The new version of https://planet.gnome.org/ looks fantastic! I wonder if we could have a similar overhaul for https://planet.debian.org/
@Ganneff Not sure if it is as simple as providing a CSS. I don't think they use the same tool to render the html.
They do provide a link to their planet repository. Maybe someone more knowledgeable than me with the planet software want's to have a peek: https://gitlab.gnome.org/Teams/Websites/planet.gnome.org/
@Ganneff Turns out that planet.debian runs on Planet Venus (https://github.com/rubys/venus/tree/master), a software that is unmaintained for more than 10 years. Maybe this is a good opportunity to migrate to the software gnome uses? Similar to Venus, their software is Python based, and runs on gitlab-ci -- so we could even integrate it using Salsa.
@Ganneff @wouter I tinkered a bit with gnome's planet software and it seems to be compatible with debian's. The ini format for the roster is compatible and there's also a "heads" directory for png's.
Compatibility aside, it is also slightly buggy. Timeouts are not set and it has issues with parsing datetimes properly. But these are only small issues, the code is only 200 LOC Python and easy to understand, so it could be a good starting point to use this.
@venthur @Ganneff @wouter I’m a bit scared by this:
runs on gitlab-ci
Please make sure it honours the <ttl> of individual feeds. Some have lower ones, some have higher ones. Some feedreaders flood my access_log with requests much more frequent than the TTL (though I’m not currently in Plänet Debian, no need to spread this bug further).
I've never heard of this attribute.
Can you give me an example of a feed with this attribute? That'd make it easy to add a (currently failing) test case which I can then implement properly.
Preferably as a bug report against the salsa repository.
@venthur @Ganneff
@venthur @Ganneff @wouter www.mirbsd.org/news.rss probably (sry omw to bed rn)
Checking https://www.rssboard.org/rss-specification#ltttlgtSubelementOfLtchannelgt, I'm not sure that ttl actually means "do not update more often than this"; instead, my reading is that it means "if the source is older than this value, invalidate the cache entries for them". So I don't think it's a bug to update at every run...
@venthur @Ganneff
@wouter @Ganneff @mirabilos Also, RSS is not the only format available. It is probably simpler to keep the crawler frequency reasonable and rely on some kind of HTTP caching to minimize traffic on the servers.
PtLink uses a format-agnostic parser to figure out details of a stream, so that's not an issue (it will just return undef if a valid is not set, whether because it's optional and not set or because the format doesn't support it).
It also uses HTTP headers such as if-changed-since (not sure about etag, might have that too) to tell the server not to send anything unless there's an update since the last change.
@Ganneff @mirabilos
- replies
- 0
- announces
- 0
- likes
- 1
@wouter @Ganneff @venthur if one feedreader queries every 15 minutes, it’s not so much a problem, but if several dozens do…
Wouter, I read this as the reader should cache for that long. (Perhaps not "for at least that long", sure, but is there a better way to express this? And it’s close enough anyway.)
I read it as "don't bother using your cache if it's older than this"
Unfortunately the spec is not very detailed, so while I think my reading is correct, I'm not confident enough of that to state that yours is wrong.
Regardless of which is the intended reading, it could definitely benefit from a "don't bother asking me if your cache is more recent than this" field.
@Ganneff @venthur
@venthur @Ganneff Oh yes! https://salsa.debian.org/wouter/ptlink. It's in production use for https://planet.grep.be, and I have a (somewhat outdated by now, but still) configuration that works for Planet Debian at https://salsa.debian.org/wouter/pd-config/-/tree/ptlink?ref_type=heads
@venthur
Forgot to mention:
Planet Debian uses Django templates, which PtLink does not support (because perl, and Dotiac::DTL doesn't actually work as advertised -- yes, I tried). So I translated the templates to perl. Since Planet Debian has templates in a different git repository, so do I ;-)
https://salsa.debian.org/wouter/pd-templates