-
-
Notifications
You must be signed in to change notification settings - Fork 401
rss-bot: Support unordered RSS feeds #895
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
rss-bot had 2 different feed_file variables: 1. The user provided file with the list of feed URLs. 2. The file for each feed URL, to store the feed entries' hashes. To clearly differentiate between them, the latter has been renamed to feed_hashes_file.
Previously, it was being set for every entry.
Renamed the OLDNESS_THRESHOLD constant that was being used for the same, to match the name of the newly added option.
We will be using entry time for sorting entries in the following commits.
|
@Pritesh-30 Could you please help me manually test this PR?
And let me know if you can find any missing edge cases. |
By splitting the logic into two loops - one for processing all the entries in the feed, and another to post only the latest ones in chronological order. Instead of tracking new_hashes in memory while processing the feed file, we track unhashed_entries now, since we will not be hashing all the entries, only the ones that we post. Fixes zulip#831.
@Niloth-p Sure, I can help with the manual testing. I’ll run this locally against a few feeds and the cases you have mentioned & report back with the results once I’m done. |
|
@Niloth-p I manually tested using real RSS feeds. I used the following feeds for testing: The entries were posted to the I tested for :
Observed behavior:
One edge case I encountered:
Overall, behavior looks correct from manual testing. |
|
Thank you for testing, @Pritesh-30! (Marking this as draft because I want to discuss a couple of design decisions on CZO before proceeding.) |
|
@Niloth-p It happend with this feed https://uptime.com/rss. I looked into it, it says feedparser handles if title is present and empty but not if the title tag is is not present. |
Fixes: #831
I've tested the basic functionality (unordered feeds + max batch size), but I haven't tested every case thoroughly (yet).
Self-review checklist
(variable names, code reuse, readability, etc.).
Communicate decisions, questions, and potential concerns.
Individual commits are ready for review (see commit discipline).
Completed manual review and testing of the following: