pleroma.debian.social

Anyone have a program they can recommend that can download an entire account's public posts? I want to backup my old posts from Polymaths Alpha but GoToSocial doesn't have post exports (yet)

@roguefoam I think the best people to ask would be @rl_dane and @amin who would likely have a bash script for this.

@orbitalmartian @roguefoam @amin

Well, there's toot timeline -a ${MyAccount} -c 20, but that only gives you 20 posts (which is the maximum).

You could also curl or $TEXTBROWSER -dump your public timeline, but that's still a limited number of posts, and only public ones.

Best bet would be to read up on the mastodon API in python that toot uses, and see if you could make a dumper program.

@amin, any ideas?

@rl_dane @orbitalmartian @roguefoam @amin a bit of SQL could give you all posts, public or not. If you need to take the visibility into account, that’s also easily done.

gotosocial=> SELECT id, in_reply_to_uri, text
 FROM statuses
 WHERE boost_of_id IS NULL
   AND account_id='01………'
   AND visibility IN ('public', 'unlocked', 'followers_only')
 ORDER BY id;

This shows all but 'direct' messages. Amin could likely export those for you. Another useful column is created_at, FWIW.

I find the default output a bit hard to parse, you may wish to use a CSV: psql --csv -c "sql code here" gotosocial >dump.csv

Otherwise… python3-mastodon isn’t that hard to use, but I’ve had issues with the mstdn-compatible client API that all those clients use, specifically, it likely won’t show posts behind some cutoff point, just like the blue birdy didn’t.

@mirabilos @rl_dane @orbitalmartian @amin I was going to say...I'd need Amin to pop into the database for that one :D We'll see what he says when it's not dawn for him though...

@roguefoam @rl_dane @orbitalmartian @amin at least I already have sufficient SQL and am at hand for extra questions if needed. (Pays off to have had @cloud_manul give me one-on-one deep SQL training.)

@mirabilos @rl_dane @orbitalmartian @amin @cloud_manul I've still gotta sit down and learn SQL - I rely too much on interfaces and packages (these days, 99% of the time the SQLite handler that Django has built-in), I'd rather know how to do it myself but never have the time.

I do have some old SQL puzzle books I was hoping to go through one of these days, maybe I'll find some time in 2025 to do them.

@roguefoam @rl_dane @orbitalmartian @amin @cloud_manul ugh, objectrelational mappers are abstractions that many consider harmful (for all but the most basic CRUD things, which Django is good at) these days.

Incidentally @ Amin I added me a second user to the GtS database, which I can connect to as regular user on my server, and granted that only reading rights on the tables, so I don’t accidentally write/delete anything. (Still got to preserve privacy on nōn-solo instances of course, but I figure post visibility 'public' is likely good for testing the commands.) Commands on request.

I personally would recommend to stay off SQLite for legal reasons and MySQL/MariaDB/MaxDB for reasons of sanity (even MS SQL is much closer to standard SQL). PostgreSQL is good. Horracle… has its uses, but why write SELECT 1 FROM dual; when you can just write SELECT;? ☻

@mirabilos @rl_dane @orbitalmartian @amin @cloud_manul Trust, I'm only using SQLite because my laptop is incredibly low on storage space to the point I worry about booting up containers for dev work (working on that front over the holiday though...) - likely as soon as I move this app I'm working on to a prod server I'll just switch the DB over to Postgres because Django can kinda just hotswap like that.

@roguefoam @rl_dane @orbitalmartian @amin @cloud_manul PostgreSQL doesn’t use that much more for the data files but the WAL can grow to excessive amounts. I reduced that on my instance server as well (it’s in the backups and I am okay with 2×32 MiB extra usage but not 2 GiB, every day).

(I still do SQL plaintext backups of the DB as well. I may never get over that, plus they’re dead useful for partial restores and searches.)

@roguefoam @mirabilos @orbitalmartian @amin

This is what I use to try to stay TZ-appropriate: ;)

worldclock () { 
    local fmt="%-10s %s\n";
    printf "$fmt" "Dallas" "$(date)";
    printf "$fmt" "Las Vegas" "$(TZ=America/Los_Angeles date)";
    printf "$fmt" "London/GMT" "$(TZ=UTC date)";
    printf "$fmt" "Singapore" "$(TZ=Singapore date)"
}

(Obviously, populate it with the time zones that are important to you)

(also, obviously the #AOSP clock app on my phone)

@roguefoam @orbitalmartian @rl_dane @amin I just consider Fedi a lossy asynchronous medium. Some things have a day or more RTT, but that’s okay.

@wouter @cloud_manul @amin @orbitalmartian @rl_dane @roguefoam it’s not a bad thing, just… not that big ;-)

@mirabilos @roguefoam @orbitalmartian @amin @cloud_manul

SQLite's licensing is bonkers.

Like, was the MIT license seriously not milquetoast enough for y'all?

@rl_dane @mirabilos @orbitalmartian @amin @cloud_manul That is true...I always forget how insane it is.

@roguefoam @mirabilos @orbitalmartian @amin @cloud_manul

I'm trying to be more mindful of not lambasting FOSS projects on the fediverse, because they often do read stuff, and they deserve respect for their efforts... (unlike, ya know, this ranty little bugga I posted a few days back )

...but scubrats, why in the world do you make your own devs jump through bizarre legal hoops just so you can make sure you require the absolute minimum of your users? I don't really get it.

@rl_dane @roguefoam @orbitalmartian @amin @cloud_manul not to mention that all these hoops don’t even work in most countries of the world, and their own offer of a 1000$ alternative licence invalidates their PD statement even over there (or is an admission at committing copyfraud).

@rl_dane @roguefoam @orbitalmartian @amin @cloud_manul so, technically, it’s not even FOSS for two distinct (if related) reasons

@amin @cloud_manul @roytam1 @orbitalmartian @roguefoam @rl_dane I can avoid using it in my own software though.

@mirabilos @roguefoam @rl_dane @orbitalmartian @cloud_manul

Yeah, I think I could figure out the SQL of it without too much issue. I've been asked questions along these lines often enough though I think I'd rather give the first shot at an API-based solution that doesn't require me to frequently dive into the database manually.

@amin @orbitalmartian @cloud_manul @rl_dane @roguefoam yeah, well, I don’t have much trust in the API. But perhaps do both and compare.

@mirabilos @roguefoam @orbitalmartian @amin @cloud_manul

I'm not sure; I think the fact that it's PD invalidates the silly $1,000 license deal altogether, but then again, it being PD is more of a defence to use in court, and not as much an ironclad legal status, but I'm just speculating.

I'm thinking that a license like CC0 would still be counted as FOSS.

This FAQ from the OSI seems to cast some doubt on the "FOSSness" of CC0 in particular, but it's not an explicit denial.

@rl_dane @mirabilos @roguefoam @orbitalmartian @cloud_manul

I'm thinking that a license like CC0 would still be counted as FOSS.

Depends. iirc Fedora has stopped accepting CC0-licensed code.

@cloud_manul @roguefoam @rl_dane @amin @orbitalmartian it’s the other way round, the fact that they commercially distribute 1000$ licences invalidates the PD dedication (or rather, what of it was valid anyway considering they have contributors from legislations (such as Germany) where you cannot relinquish copyright and place works into PD). The Berne Convention also defaults to protection in all other signatory countries.

CC0 is OSD-compliant but the explicit not-licencing of patents is why OSI hasn’t approved it and why Fedora isn’t accepting it as a licence on software any more (it’s still fine for data, docs, artwork, etc. to them). I’ve begun to dual-licence (MirBSD or CC0) all works I had previously put under CC0 due to that.

CC0 is interesting. It has a PD dedication (§2), which is good as never valid in international scenarios, and a fallback licence (§3) which doesn’t licence the work; rather, it allows recipients to licence the work. This is possibly scaring, but given people invent licences like "0BSD" that don’t require disclaimer reproduction, they seem to like living in danger of product liability (instead of however much they can limit liability for FOSS works).

@rl_dane @roguefoam @orbitalmartian @cloud_manul @amin (see other reply to the reply here)

I see you replied to the technically post. Well, technically PD isn’t FOSS, even if (correctly exercised, i.e. author dead for 50/70/100 years depending on legislation) it has no practical difference.

@roguefoam @mirabilos @rl_dane @orbitalmartian @cloud_manul

Is said laptop Linux? This might help if you haven't run it before:

# journalctl --vacuum-time=2d

That'll remove JournalCTL logs older than two days. They just tend to build up endlessly if you don't clear them every now and then. I just ran it and cleared 360MB, and I clear it regularly.

@amin @mirabilos @rl_dane @orbitalmartian @cloud_manul Mac, so Unix but not with journalctl unfortunately. I've just been manually clearing out files - spent about an hour and got 12GB deleted and 1GB moved to my school folder to be moved off to an external drive in a bit here.

@roguefoam @mirabilos @rl_dane @orbitalmartian @cloud_manul

Ah, okay.

@roguefoam @mirabilos @rl_dane @orbitalmartian @cloud_manul

(btw "not with JournalCTL" is not an "unfortunately" imo. 😉)

@amin

Vacuuming done, freed 976.1M of archived journals from /var/log/journal/adb0686b21b940ce94bd1c17d9200161.

Wow, almost a gig. I actually had no idea that these logs ate so much space. Thanks for the hint!

@82mhz

Yeah, they're pretty annoying.

@82mhz @amin yet they still corrupt on crash…

@mirabilos @roguefoam @rl_dane @orbitalmartian @cloud_manul

That's not a bad idea, I should probably add said second database account.

@amin @mirabilos @roguefoam @orbitalmartian @cloud_manul

Just curious, would the DB account give access to direct messages?

@rl_dane @amin @orbitalmartian @roguefoam @cloud_manul yes, naturally

@rl_dane @roguefoam @mirabilos @orbitalmartian @cloud_manul

Not to speak of the fact that in many jurisdictions (namely Germany, iirc) public domain declarations don't actually work, meaning it reverts to "all rights reserved".

@rl_dane @amin @roguefoam @orbitalmartian @cloud_manul IIRC most of the EU (not UK though, common law vs. the other thing, but they’re out now anyway) and quite possibly other countries, I only know it for sure for Germany though.

What’s more, the Berne Convention says a work from country A is protected in country B as if a resident of B made it, by default, and has no concept of waiving copyright manually IIUC, so if something is PD in any country, it’s still protected in all other signatories (almost all countries); a lawyer from USA (with a side note of "most PR dedications aren’t even valid in USA" due to wrong form etc.) on the OSI ML said the same.

In the case of "PD in USA by law (government work)", someone (govt employee) has successfully defended their copyright on the work in a court in Europe, FWIW, so this may not be very much off.

Now there’s analysis from german lawyers that, with a PD dedication, authors have insufficient motivation to sue anyone over it in even Germany, the courts would likely laugh them off.

But that’s just looking at the "can I get sued over it" side, missing the "is it properly licenced FOSS" side. Using something like that (remixing) in another work is still not only not permitted (as is distributing it), it also endangers said other work.

Commercial copyright violation is also a felony. I’m not sure whether this actually needs the author to sue you or if the state can do that, in which case the "the author can’t sue you over it" becomes much less important suddenly than it was when "only the author (and their PRS) can sue over copyright violations".

IANAL, TINLA. But I’ve been writing people asking for fallback licences for PD stuff in BSD and mostly gotten positive replies.