/hydrus/ - Hydrus Network

Are there any ways the Hydrus "team" could make updating a bit more foolproof?

Currently, it is not recommended upgrading past 10 versions.

I was thinking, could there not be some sort of implementation that instead of allowing a user to upgrade 10 versions apart, it does it incrementally automatically? For example, if a user is on version 10, and they upgrade to version 20, it instead doesn't try to directly upgrade to version 20, but instead upgrades to version 15, then does 20. This would be very nice, to make it a bit more foolproof.

I know it gives the warning, and says not to, however it would be nice if it could be done automatically.

_____

My other suggestion was having a RSS feed on the Hydrus website, that posted suggested updates for big fixes, or major security fixes. The releases usually have massive changelogs, and they get pushed fairly often, so I don't blame people for falling behind on updates. It would be nice to be able to subscribe to a Hydrus feed that gives suggestions on when you really should update Hydrus in the case of major security or bug fixes as these are important.

>>14959

The alternate system isn't developed enough yet to be useful for things like that. You can't sort by alternates.

So for now the best way is still using tags like title and page numbers.

>>14915

This, tagging now is so annoying.

Unfortunately, dev has a tendency to leave things 90% finished…

>>14915

This, tagging now is so annoying.

Unfortunately, dev has a tendency to leave things 95% finished…

Is there an easy way to check for and re-download broken files? Basically I want a button that regenerates all the thumbnails and if a thumbnail fails go to one of the source links and re-download a good copy of that file.

Background: I was scrolling through a page with a couple thousand files, got a couple messages about a thumbnail not generating. It was annoying to find the files in question when only given the on-disk file name and even after finding it on disk actually finding it in hydrus is still a pain because afaik you can't search by hash (?). Then even if you find a broken file in hydrus, if the file is actually messed up there seems to be no way to fix it besides manually downloading the file externally and replacing it on disk. We already check if files are valid and most have a list of sources so this seems like most of the parts are there?

Is there a way to get notifications of rejected petitions? It seems like they act as permanent client-side deletions (or sibling/parent changes) that notify the repository.

I think local deletions should be separate from petitions. If one gets rejected I'd rather stay in-sync with the PTR and resubmit or do a local deletion if necessary.

as of now, can Hydrus only support grouping image sets/manga page by setting the images all as "alternates?" I can't find any other options that to that effect.

Not sure if this deserves to be its own thread so I'll just ask here. When you make the polls about which major jobs you'll work on next, do you allow multi-choice voting?

To make it short, multi-choice voting is better for polls like this, because it lets people express approval for multiple option if they think that more than one of them would be an acceptable winner, and this leads to the winning option naturally being one that more people like overall. I checked the github repo for the major jobs, and I can already see a situation where I'd have to not vote for one of my favorite options present because it's probably going to be less popular, and I'd have to pick one that's more likely to actually win. I won't be able to show my support for both.

poal.me (for example) has an "Allow multiple choices per voter" option, if that's what you use to run the polls. I'm a little new here so I don't know if that is what you use, but it's popular on imageboards.

forgot to clear trash and ended up with no hard drive space when subs were running. Now Im getting

>sqlite3.DatabaseError: database disk image is malformed

There a way to remove the stuff that was downloaded the last 1-2 days to fix this? Cant even open hydrus to delete cus of this message.

>>15021

nm fixed by following instructions in the

>help my db is broke.txt

to sql clone the dbs

Hey fellas. I've lost the drive that my files lived on, and all that remains are the thumbnail files. Is there any quick way to re-download the files via the filenames of the thumbnail files with a gallery download? They seem to be too long to be md5s, which would be an easy job. Thanks, and congratulations on the anniversary!

>>15027

I'm pretty sure that those are the SHA256 hashes of the file they are a thumbnail of (not 100% sure though). I doubt this that would be very useful for re-downloading them. It may be possible to get the URLs for your files that were downloaded within Hydrus if your database is intact, you'll have to ask a database expert on that though.

Enjoy your empty inbox, and consider investing in backups this time :^)

>>15028

Yeah lesson learned. I had a backup, I think, but it may have been kept on the same drive as the collection was. Oops.

Gallery downloads stop downloading if I tab to another page besides gallery in v402. Is there a setting to change this or am i missing somethin?

>>15034

Nm it looks like Im getting bandwidth free in x hours which probably has something to do with it…

Also getting bandwidth free in x hours every time I come back. Doesnt make sense becaus default bandwidth rule is not exceeded for any of the network contexts.

>>15035

ok I wasted a ridiculous amount of time on this bcuz I didnt remember to set new bandwidth rules for the domain too, not just the 4-5 default contexts or whatever. Had to double click the domain name in the bandwidth manager to get the menu

A lot of downloaders suddenly broke for me. For one site the gallery downloader works but it gives an instant "Connection refused" error from urllib3 when a file is attempted to be downloaded (the URL of which opens fine in a web browser), for others even the gallery is refused.

Some don't even seem to be trying to download anything at all, not even enough to give an error on (maybe Hydrus automatically stopped trying after too many errors, though a few of these were sites I've never used).

A few downloaders still work perfectly fine, though.

If it matters, here's all that I could test.

>everything working: lolibooru, pixiv tag search

>only gallery working: sankaku

>nothing working: danbooru, gelbooru, konachan, realbooru, e621

Anyone know what could be going on?

How do I tell Hydrus which font to use? Is qt making the decision?

>>15040

You can set fonts via qt stylesheet's located in your root hydrus directory in static/qss

Is there a script to use or fix the blacklist for gelbooru when using the downloader? For some reason hydrus ignores my blacklist over on gelbooru and continues to download the stuff I don't want.

Is it possible to skip ahead or speed up videos in the media viewer? Ideally, I'd like to be able to add/change shortcuts for it, but it doesn't look like that's possible at the moment.

Is it possible to have the database itself on one drive, and the client_files themselves on another?

I fucked up my duplicates. is there any way to reset both applied duplicate pairs and potential duplicate pairs?

oh also what happens to the files that are deemed worse or equal? the help says it is kept in a duplicate file group but can it still be found in searches and is it still on my computer?

>>15053

Make sure you have your tag_blacklist cookie from gelbooru in hydrus.

>>15058

Not currently

>>15063

Yes, see https://hydrusnetwork.github.io/hydrus/help/database_migration.html

>>15064

You can reset all potential duplicates if you click the gear icon in the duplicates tab. I am not sure if there is an easier way to reset applied duplicates but you can select the files then right click and reset them under the manage>file relationships menu.

>>15065

Worse are moved to trash, equal are both kept. Files in trash do not come up in normal searches and by default are removed from the system periodically.

>>15066

>Yes, see https://hydrusnetwork.github.io/hydrus/help/database_migration.html

thanks

god fuck it's so much faster now

>>15069

> I did import the new header from here due to not being able to see explicit tags.

That may be the case, try removing the header for gelbooru and adding the fringeBenefits=yup cookie into the cookies menu for gelbooru.

I'm not sure if this is the right place to ask, but I can't watch or download from 8chan. I keep getting 404 or not found. Is there something I'm missing?

Is there any way of making downloading from twitter practical? The downloader keeps dropping the connection with 404 results when getting galleries and this makes the downloader think that it's done with the job. This gives me incomplete gallery downloads. Is there a bypass/fix to make the downloader at least fail with another status code besides DONE so I don't accidentally think it has actually downloaded everything?

is there a way to download my private pixiv bookmarks? I used the downloader and it grabbed my public bookmarks, but some of my older bookmarks are marked private

>>15071

I literally just came here to ask the same thing. If you find a workaround, please post it here.

>>14731

Is there any way to force hydrus to import one picture at a time in order? That way a set of images can be sorted by import date. Any other way to achieve the same? Now, let's say I have three sets of images that I import at one time, it might import, let's say second set's first picture before first set's last. I could add page numbers via regex, but I think that's a bit clunky way to achieve this. Do I make any sense? Thanks.

>>15070

Dumb question but which or where is the header?

>>15103

network>data>manage http headers

>>15115

are you running from source? you need python-mpv.

>>15119

in the tag manage menu there will be multiple tabs, tags will only be added to the public tag repo if you enter tags in that tab. So just don't add those tags in that tab.

>>15122

That's just gelbooru changing the site and breaking the parser, see >>15118

Hey, I am sorry, the holiday ended up killing my schedule, and I couldn't give this thread proper attention.

8kun are now saying the expect to go off the clearnet in the near future and go TOR only. I have been wanting to move to a different board for a while, I was just procrastinating, so this seems like the time to do it. Endchan will be our primary board for the time being. I am going to clear this thread of newer posts, make an archive of it to post on Endchan, and then make a sticky announcing I will delete the board next week.

>>14908

>>14914

Not yet. In future I want better options and other portability.

>>14915

>>14977

>>14978

Sorry for this. This last 10% ended up needing more additional work to get my taglist supporting the additional rows to show parents, which was more complicated than I thought.

Several jobs are waiting on taglist presentation improvements, including tree-like [+] buttons to collapse/expand when sorting by namespaces. I am not sure when I will find the time, but it is still on my list.

>>14921

I am sorry, my internal backup solution is not clever enough to deal with a migrated database. I have a job to make this more clear in the UI and help. Since you are an advanced user, please set up a backup routine with an external program that can do the job better than I can. I recommend FreeFileSync:

https://hydrusnetwork.github.io/hydrus/help/getting_started_installing.html#backing_up

>>14935

No way at the moment, it is just a patch for advanced users. Reducing CPU overhead, particularly with large downloaders in the current session, is the next primary optimisation job.

>>14946

>>14957

I think the default DA parser does not pull artist tags because they are so non-useful and I don't want new users to get them. I know nothing about this one, but maybe it gets what you want?

https://github.com/CuddleBear92/Hydrus-Presets-and-Scripts/blob/master/Downloaders/deviant%20art%20page%20parsers%20-%20fetch%20tags%20-%202020-07-18.png

>>14968

Unfortunately I am the only person working on hydrus, as I have acute difficulty working with others. Since I am always short on time, and I am not always as professional as I hope to be, I cannot guarantee nice user experience for certain parts of the program.

>>15008

I think database->maintenance->files->manage->add new work->if file is missing/incorrect and has url, move file out and try to redownload is the basic route you want. This assumes the files have actually been damaged since import, though (e.g. hard drive damage), rather than them once being importable, but technically no longer allowed with stricter rules now about how broken a jpeg will be accepted into hydrus.

Try adding a 'my tags' tag to those files you saw and adding that file maintenance job just to a search for that tag. It might fix them, it might not. I had to turn up the safety limiters on truncated jpegs about 6-12 months ago because it was crashing some clients on really broken files.

>>15012

Not yet. I don't think I will be able to deliver it this round, but I want to begin the framework for this in the network improvements I am starting properly this week. The PTR jannies want to give reasons back for denied petitions too.

>>15017

Not yet. When I get to alternates improvement, you will. It came #3 in the latest big jobs poll, so I think I should get to it within 12-15 months.

>>15027

>>15028

>>15032

If it helps, if you still have this problem at all, you want the same file maintenance job that I just said in the post above. >>15128

>>15029

I am sorry for the trouble here. The blacklist should work on the tags parsed from the page. So if you tell it to stop 'guro', then any page with that tag should get the 'ignored' status with an appropriate note in a downloader using those 'tag import options'.

One user has been telling me about this not working for him, but I couldn't figure it out. Can you give a search example on danbooru and a blacklisted tag that aren't applying for you?

>>15039

I am sorry, I do not know what is going on here. Sounds like it could be an ssl issue or similar, maybe if your OS is old, although I'd guess that sort of thing would be more likely right on Jan 1st 2021 as some certificate timeout ticked over.

If you can paste the whole error, I can have a closer look. If it is moaning about ssl certificates, one way around it is to uncheck the 'verify regular https traffic' option under options->connection.

>>15071

>>15089

If you mean 8chan.moe, I believe they did some direct linking referral trickery on their spicier boards to make sure direct links from external sites don't work. There may be a solution here with some forced hydrus referral urls on the URL Classes, but I do not know the details and have not looked into it myself. I think you'll have to browse these boards manually for now.

>>15074

Any site that doesn't operate like a normal booru is tricky. Sometimes you just have to babysit them.

I assume you are downloading through nitter, right? I think the twitter API that nitter hooks into does not deliver tweets beyond about 3,000 tweets into the past, so if you are hitting that limit somehow, or if nitter has its own limit, that may explain it.

I have been getting reports recently that nitter has been giving '429 - server rources limited' or similar recently as well. This may be related, I am not sure.

>>15088

I am not sure if the existing pixiv login script works right now, it has always been temperamental. If not, I think you'll have to use Hydrus Companion or similar to log in by copying your browser cookies to hydrus. Then pixiv will think your hydrus is you and it'll have access to private stuff.

>>15095

Support here is not yet great. If you are importing from hard drive, they should import in filename order. If you are importing from a site, it will come in the order the site's parser pulls the URLs, which is usually good, sometimes reversed. The main problem is hydrus currently has integer timestamp for import time, so if two files are imported in the same second, they will sort randomly when you sort by import time. If this is what is hitting you, I apologise. At some point there will be a conversion to sub-second times, although a retroactive fix would likely be impossible.

I am hoping the eventual file alternates work will provide a nicer system than page tags. I agree they are clunky.

>>15115

>>15116

This is some developer bullshit unfortunately. You don't need mpv, but 'libmpv1'. You should be able to get it with apt.

This thread is now archived at https://archive.is/0WoQk . I will not post here any more, as the board will be deleted next Wednesday. Please move to the Endchan thread at https://endchan.org/hydrus/res/10.html , thank you!

The plan to move to Endchan only has changed!

Codexx on 8chan.moe has kindly offered to host me on /t/ over there. As I have never been a comfortable BO, this is a much better solution for me as the primary place to talk Anon about hydrus.

I will be maintaining a Hydrus Network General, the first instance is here: https://8chan.moe/t/res/2219.html

The links in the release tomorrow will reflect this. Endchan will remain as a bunker.