Changelog 200-249

version 249

reintroduced shape and colour options to edit ratings service panels
reintroduced num_stars and allow_zero options to edit numerical ratings service panels
the export phrase--defaulting to '{hash}'--will now persist through export dialogs. it is saved whenever you click update or export
removed the unintended 'counts' that were appearing after related tags
fixed 'remove' action in custom filters
hitting the delete key on manage tags taglist now will always remove
added a BUGFIX option to not verify regular https traffic on the old networking engine for those non-Windows users who are getting SSL verify errors
refactored the top hover frame more, making it more flexible
created a new top hover frame for the duplicates filter
duplicates filter now also supports tag and ratings hover frames
duplicates filter now reports A or B as file index
added some placeholder buttons to the duplicates filter for actions and 'cog' customisation
reduced duplicate search shutdown logspam
improved duplicate search job cleanup
improved mime detection of thumbnail regeneration
improved some shutdown error handling
improved numerical rating search accuracy
improved some hdd import error handling
improved some unicode error handling
improved reliability of some database transaction processing
fixed animationbar having problems with single-frame videos
fixed namespace colour taglist not updating colour correctly
stopped the namespace colour taglist from deleting default namespace colours
made the new class of button more compact--not sure if I like it
refactored some db initialisation to avoid future transaction problems during special updates
tidied up some last incomplete taglist code from last week
did a little prep for some future shortucts overhaul
did some more menu code updating
added a simple subscription save/load unit test
moved outdated server unit tests forward
fleshed out some server certificate generation
some other misc v245 catchup work
cleaned up some pydeadobjecterrors caused by downloaders reporting progress to destroyed windows

version 248

fixed two more issues with recent update code!
re-applied a more rigorously tested 'invalid subtag' replacement for any user that has experienced incomplete update code in the past few weeks
added some tag rendering options to options->tags that affect most user-facing, non-editing windows:
added custom namespace-subtag connector (i.e. instead of ':')
added 'show namespaces' checkbox (i.e. to keep the colour but lose the namespace, like how most boorus display tags)
created a basic framework for the duplicate filter--it doesn't do anything yet, but you can launch it from the duplicates page to check it out
updated 'reset processing cache' action and added it to review services repository pages
review services now always starts on 'my files'
review services will try to stay looking at the currently selected service through a refresh_services event
fixed some thread-gui refresh interaction in review services
all listctrl-attached delete/remove buttons will now ask if you are sure before they go
reenabled db->maintenance->clear orphans, which it turns out I had already fixed by accident
serverside services will lock themselves more efficiently while generating updates, ensuring 'server busy' responses can always return promptly
fixed tag manage parents a/c entry to remove tags when they already exist in the box (previously, the a/c would only ever add, so a double-click was required to remove)
tuned adminside petition processing so it approves/denies in reasonable-sized chunks
refactored all the taglist gui code to focus on the underlying tag rather than the string representation
plenty of misc taglist cleanup besides
started a unit test suite for listboxes and wrote a thorough test for namespace colour listbox
misc canvas refactoring

version 247

fixed a problem with deleting more than 256 files at once
furthermore, deleting from thumbnail view or after a filter will split delete jobs into chunks of 64 files at a time to reduce gui hang from deleting many tag-heavy files
rewrote canvas media container code to recycle containers, embed windows, animation bars, static image windows, and animation windows. scrolling through all kinds of media is less flickery (less 'grey box' window initialise flicker) and scrolling through static images should be completely flickerless! William Gibson slideshow speed works again!
this flickerless static image transition will be particularly useful in the forthcoming duplicate image filter!
got adminside petition processing working again
petition counts and fetching is now split by content_type and status
the approve/deny colour hint is more obvious on the petition panel
petitions now process off the main gui thread and throw up a popup message
added 'check all' and 'check none' buttons to petition panel
several serverside petition processing fixes
generalised and improved dynamic menu check item initialisation and inversion support
moved the 'get tags even if file already in db' option into a cog button on regular downloader pages
added a default menu option for 'get tags even if file already in db' to the same cog button
added this cog button to the edit subscription panel as well
fixed exporting tags to Hydrus Tag Archives
fixed exporting 'all known tags' to HTAs
cleaned some HTA and related code
fixed namespace based tag censorship
fixed autocomplete not filtering out current/pending counts if they are set to 'excluded'
fixed generation of non-expiring new accounts
fixed some v245->v246 tag improvement code that was replacing invalid tags with the incorrect namespace
patched a problem with '-:' dirty tag in the v245->v246 update code--I'm not sure what it was doing, but it catches the unusual problem and puts it in the 'invalid tags' category, so let me know if you get trouble with this in future
wrote unit tests for bytesdictionary and shortcuts serialisable objects
harmonised and improved how separators are appended to menus
cleaned up some client db index creation
cleaned up some client tuple stripping
misc pylint warnings cleanup
misc fixes

version 246

fixed a critical bug in serverside content deserialisating that meant servers were not processing most client-submitted data properly
fixed a critical bug in 'all known tags' autocomplete regeneration--please run database->regen->a/c cache when it is convenient and let me know if your numbers are still off
fixed the pre-v238 update problem for good by abandoning update attempts and advising users to try v238 first
clientside invalid tags will now be collapsed like with the server last week. if a tag is invalid (typically something with an extra space, like "series: blah"), the update code will attempt to replace existing mappings with the collapsed valid version. some unusual cases may remain--they will be replaced with 'invalid namespace "series "' and similar. Please remove and replace these when convenient and contact me if there are way too many for you to deal with
duplicates pages now have a file domain for the filtering section, and they remember this domain through session loads
this file domain is accurate--counting potential duplicates and fetching pairs for 'show some pairs' only from those domains. the issue of remote files appearing should be gone!
there is now only one 'idle' entry in the duplicates page cog menu--it combines the three previous into one
fixed numerous irregularities across the wildcard code. all search input now has an implicit '*' on the end unless you put a '*' anywhere else, in which case it acts exactly as you enter it, with a non-* beginning matching beginning of string, whitespace, or colon, and non-* end matching end of string or whitespace
autocomplete now searches namespace, so entering 'char' will load up all the 'character:' tags along with 'series:di gi charat'. this can lag to hell and back, so it may either need some work or be optional in the future. feedback would be appreciated
typing 'namespace:' will include all the series tags below the special optimised 'namespace:*anything*' tag
autocomplete searches recognise an explicit '*' no matter where it is in the entry text. typing 'a*' will load up all the a tags and present a 'a*' wildcard option
quickly entering a wildcard entry will now submit the correct wildcard predicate ('rather than a literal 'hel*' or whatever tag)
review services panel now reports total mappings info on tag services
review services panel now reports total files info on file services
manage services's listctrl is now type | name | deletable and initially sorts by type. the strings used for hydrus service types are also improved
manage serverside services (called by server admins to manage their services) have fixed setnondupe port and name on edit service events
new popup messages will now also appear if there were previously no popup messages to display if the current focus is on a child on_top frame, such as review services (you'll now see the processing popup appear when you click 'process now' on review services)
the popup message manager now initialises its display window with a single message that is quickly dismissed. this helps set up some variables in a safe environment so they don't have to be generated later when the gui might be minimised or otherwise unusual
hid hydrus update files from 'all local files' searches
added 'media_view' entries for hydrus update files, just in case they are still visible in some unusual contexts (and they may be again in a future update anyway)
fixed 'recent tags' being returned from the database out of order
by default, 'recent tags' is now on for new users
'get tags even if file already in db' now defaults to False
file import status now allows a 'delete' action below the 'skip' action
file import status right-click event processing is more sane
fixed the new raw width/height sort choices, which were accidentally swapped
cleaned the media sort code generally
cleared out some redundant rows that are in some users' client_files_locations
namespaced predicates are no longer count-merged with their namespaceless versions in 'write' autocomplete dropdowns
'unknown' accounts should now properly resync after clientside service change
improved how registration keys are checked serverside when fetching access keys
fixed a v244 update problem when unexected additional tag parent/sibling petitions rows exist
improved my free space test code and applied it to the old v243->v244 free space test (it'll now test free space on your temporary path and report problems appropriately)
to improve log privacy and cleanliness, and to make it easier to report profiles, db/pubsub profiles now write to a separate log file named appropriately and labelled with the process's start time
profiles are more concise and formatted a little neater
across the program, three-period ... ellipses are now replaced with their single character unicode … counterpart (except to the console, where any instance of the unicode ellipsis will now be converted back to ...)
cleaned up some log printing code
cleaned up some experimental static serialisation code, still thinking if I like it or not
started on some proper unit tests for hydrus serialisable objects
fixed and otherwise updated a heap of unit test code to account for the v245 changes
cleaned up a bunch of old database table join code
started some databse-query-tuple-stripping code cleaning
deleted more old unused code
misc timing improvements
misc code cleanup experimentation
misc cleanup

version 245

fixed a v244 update problem for clients updating from<v238< li=""></v238<>
some misc stuff:
if you start editing many subscriptions, cancelling a single dialog will break the chain of loading new dialogs
reduced some redundancy in regular client file import
improved how the dialog for selection from a list of strings works
if the client ever merges one directory to another (such as in external file locations migration) and any files fail to merge, the source will no longer be deleted
created new flexible bandwidth tracking and ruling objects
updated how repository updates work, splitting the old explicit and self-contained content update package system to a new implicit definitions/content split system and an improved one-step-sync metadata propagation
updating now takes approximately 23% as much bandwidth as before
update files are now stored in client_files and server_files like any other file (client_updates and server_updates folders will be deleted on update)
the server will print update generation info to its log
unfortunately, updates cannot be converted, so a complete resync of update files is required. the smaller update size and better bandwidth controls should mitigate the problem somewhat
the server has been compacted across all content types--its mappings db file should shrink about 12%
due to new service-specific ids, server.master.db should increase in size, typically about 50%
harmonised how GET and POST/response args are built and parsed across the network
server administration initialisation is now simpler, done with 'init' registration key
leading and trailing spaces are now removed from both the namespace and subtag components of namespaced tags, meaning 'title: blah' will be collapsed to 'title:blah'. tag repositories will clean their existing tags on update
invalid serverside tags will be replaced with valid placeholders on server update
refactored, harmonised and simplified some server request code
Eris makes a triumphant return to the root welcome page ('/' request of any service) with improved self-description text as well
updated how bandwidth is tracked and overseen
all requests now consume bandwidth
fewer requests are actually constrained by bandwidth--at the moment, it is only update files and file/thumbnail downloads, as these represent the overwhelming majority of bandwidth consumption and are not at all critical to service operation--this may change in the future, but it suits our purposes for now
bandwidth tracking code is more sane across the board
the server's administration service now tracks server-wide bandwidth as it happens
service and server-wide bandwidth rules are consulted as soon as the request begins
improved some server response rendering
permissions are more flexible and content-specific
improved how file repositories check and process file requests
created a gui control for managing bandwidth rules
updated the serverside service object
the serverside service object now contains bandwidth rules and tracking
updated how the server deals with services on a db level
refactored and cleaned a ton of server db code
updated how services are edited over the network
updated how the server associates its services with its http pipeline
converted clientside server service management gui to the new panel system
updated gui code for clientside server service management
converted clientside server service management db code to the new system
improved how bandwidth errors are reported
updated the account type object
the account type object now contains bandwidth rules and a more flexible permissions system
updated how the server deals with account types on a db level
converted clientside server account type management gui to the new panel system
updated gui code for clientside server account type management
serverside account types are now fetched from a cache, reducing memory sprawl
updated the account object
the account object now contains a bandwidth tracker
updated how the server deals with accounts on a db level
updated the clientside server object
the clientside server object now contains bandwidth rules and a tracker and has improved error management, recovery, and reporting
updated how the client deals with services on a db level
improved how serverside bandwidth errors are caught
all clientside services start with a 50MB/day bandwidth limit
existing repositories will get a 250MB/month, 50MB/day default bandwidth limit, just to help us get over the hump--see the release post for info on how to get all the updates anyway
updated manage services dialog extensively
service account registration now occurs through a simple button on the normal clientside service edit panel
all services can now be renamed
updated the content processing pipeline
prepared code for future merging of file and tag repositories
added future support for pend-petitions for files and mappings and simple creation permissions for tag siblings and parents
the clientside content processing pipeline now operates inside a single database transaction, reducing a great deal of previously redundant hard drive activity
the serverside content processing pipeline now creates per-service timestamped content definitions
the clientside content processing pipeline now maintains and appropriately consults a cache for server definitions
pre-process disk cache is more intelligent
updated review services to match the new service objects
service review panels now show more information about error state and so on
service review panels should now update as soon as services change
you can now force sync repositories from review services
you can now pause/resume repositories from review services
you can now export repository update files from review services
you can now import repository update files from the services menu
repository thumbnail download now uses a cache for faster thumbnail ownership testing
culled a lot of old code and experiments
deleted all the old messaging depot code and db table cruft
removed optimised petition processing
serverside deleteorphans is temporarily disabled
hydrus client-repository relationship no longer supports news
removed the 'stats' admin service call--it will come back as a account review page
clientside clear orphans is temporarily disabled
clientside local file/thumbnail server is disabled for now
custom service 'messages' for the root page are no longer available
some things are not working yet--they will be back in soon
content presentation on review services
service reset, other advanced content service controls
ipfs controls on review services
local booru controls on review services
most service-specific panels on manage services
petition resolution
serverside account modification, including banning

version 244

updated client database to compact ( namespace_id, tag_id ) pair into a single id for storage
added some bells and whistles to the update code
added a free space check and messagebox warning before the update
updated db, service, and a/c cache creation code to reflect new schema
updated absolutely everything else in the db to reflect the new schema
for users with plenty of tags, the db should now be about 33% smaller!
unified how unnamespaced tag searching counts are totalled
unnamespaced tag searching counts are now totalled when the tags are fetched from the in-view ui media
unified how tags are split into ( namespace, subtag ) across the program
fixed deviantart gallery thumbnail parser
fixed linux session load page key event handling bug
os x can now support notebooks with zero pages open
fixed an issue where os x was losing the first page of some session loads
fixed some similar files shutdown work false positive calculation
reduced server bandwidth check period from 24 hours to 1 hour
improved calltothread scheduling under heavy load
improved scheduling of how files are physically deleted
numerous laggy temp_table replacement/cleanup
more temp_table replacement
misc efficiency improvements and general db code cleanup
misc path code cleanup

version 243

updated more menu code to the new system
finished updating all main gui menubar and thumbnail menu code
all brackets, braces, parentheses, and single and double quotes are now ignored when matching search text to tag. inputting 'mercy' matches 'character:angela "mercy" ziegler' and entering '[[[intensifies]]]' matches '[intensifies]'
'*' searches will now return no autocomplete results, rather than trying to load absolutely everything
made wildcard searching significantly more permissable--it now implicitly applies a '*' to the beginning as well as the end of your search input
the main gui's notebook pages will now be disabled while a session loads
might have fixed a first-boot layout issue--otherwise added a layout entry to the debug menu for further testing of this problem
improved progress layout on file import dialog
created a more efficient way for threads to update gui elements
the file import dialog uses this new update system and parses lots of small files much faster as a result
the edit import folder dialog will now complain if you try to put in an import path that includes the install or db directory
ffmpeg mime parsing now catches and reports audio formats without any hassle
thread downloader no longer fetches file extension in filename tag 'filename:blah' instead of 'filename:blah.jpg'
changed png default zoom rules to copy jpg--the unusual zoom as default was more confusing than helpful
updated sqlite for windows--should be a decent bit faster
deleted a lot of old code that'll never be needed again
misc refactoring
misc cleanup

version 242

optimised 'exact match' similar file queries to run a lot faster
optimised similar file queries in general, particularly for larger cycle queries
optimised hamming distance calculation, decreasing time by roughly 45%!
the similar files tree maintenance idle job will not trigger while there are phashes still to regenerate (this was redundantly and annoyingly blatting the new dupes page as soon as phash regen was paused)
removed similar files tree maintenance entry from db->maintain menu, as it can be done better from the new dupes page
adjusted the duplicate search and file phash regen progress gauges to reflect total number of files in cache, not the current batch job
all maintenance jobs on the duplicates search page will now save their progress (and free up a hanging gui) every 30 seconds
the duplicates page's cog menu button now lets you put phash regen and tree rebalancing on the normal idle routine, defaulting both to off
the cog menu can also put duplicate searching on idle time!
added a very rough 'just show me some pairs!' button to the dupe page--it is pretty neat to finally see what is going on
I may have reduced the memory use explosion some users are getting during file phash regen maintenance
wrote an unclose_page action and added it to the shortcuts options panel--it undoes the last page close, if one exists. ctrl+u will be the default for new users, but existing users have to add it under options
added ascending/descending sort choices for width, height, ratio, and num_pixels
the client can no longer talk to old http hydrus network servers--everything is now https
in prep for a later network version update, the client now supports gzipped network strings (which compress json a lot better than the old lz4 compression)
fixed gif rendering in the Windows build--I forgot to update a build script dll patch for the new version of opencv
the export file dialog's neighbouring .txt taglist file stuff now allows you to select a specific combination of tag services
if an hdd import's original file is due to be deleted, any existing neighbouring taglist .txt file will now also be deleted
the inter-thread messaging system has a new simple way of reporting download progress on an url
the handful of things that create a downloading popup (like the youtube downloader) now use this new download reporting system
sankaku seems to be 503-broke due to cloudflare protection--I have paused all existing sankaku subscriptions and removed the sankaku entry for new users (pending a future fix on my or their end)
I've also removed danbooru for new users for now--someone can fix the long-running sample size file issue in the new downloader engine
removed unnamespaced tag support from the hentai-foundry parser--maybe someone can try to fix that mess in the new downloader engine
menubuttons can now handle boolean check menu items that are tied straight into hydrus's options
menus launched from the newer frame and dialog code will now correctly display their help text on the main gui frame's statusbar! (at least on Windows! Linux and OS X remain borked!)
fixed a unicode error parsing bug in the gallery downloader
the server stop (or restart) command now correctly uses https!
the server test code now works on https as appropriate
fixed some misc server test code
misc fixes
misc cleanup
misc layout cleanup

version 241

fixed the 'setnondupename' problem that was affecting 'add' actions on manage subscriptions, scripts, and import/export folders
added some more tests to catch this problem automatically in future
cleaned up some similar files phash regeneration logic
cleaned up similar files maintenance code to deal with the new duplicates page
wrote a similar files duplicate pair search maintenance routine
activated file phash regen button on the new duplicates page
activated branch rebalancing button on the new duplicates page
activated duplicate search button on the new duplicates page
search distance on the new duplicates page is now remembered between sessions
improved the phash algorithm to use median instead of mean--it now gives fewer apparent false positives and negatives, but I think it may also be stricter in general
the duplicate system now discards phashes for blank, flat colour images (this will be more useful when I reintroduce dupe checking for animations, which often start with a black frame)
misc phash code cleanup
all local jpegs and pngs will be scheduled for phash regeneration on update as their current phashes are legacies of several older versions of the algorithm
debuted a cog menu button on the new duplicates page to refresh the page and reset found potential duplicate pairs--this cog should be making appearances elsewhere to add settings and reduce excess buttons
improved some search logic that was refreshing too much info on an 'include current/pending tags' button press
fixed pixiv login--for now!
system:dimensions now catches an enter key event and passes it to the correct ok button, rather than always num_pixels
fixed some bad http->https conversion when uploading files to file repo
folder deletion will try to deal better with read-only nested files
tag parent uploads will now go one at a time (rather than up to 100 as before) to reduce commit lag
updated to python 2.7.13 for windows
updated to OpenCV 3.2 for windows--this new version does not crash with the same files that 3.1 does, so I recommend windows users turn off 'load images with pil' under options->media if they have it set
I think I improved some unicode error handling
added LICENSE_PATH and harmonised various instances of default db dir creation to DEFAULT_DB_DIR, both in HydrusConstants
misc code cleanup and bitmap button cleanup

version 240

improved how the client analyzes itself, reducing maintenance latency and also overall cpu usage. syncing a big repo will no longer introduce lingering large lag, and huge analyze jobs will run significantly less frequently
the analyze cache will be reset on update, so you will have one big round of analyze the next time you maintain, and then you are good
added data structures to support auto-discovery of duplicate files
improved how some similar files maintenance occurs
flushed out duplicate status reporting
added a new page type, currently under pages->search pages->duplicates, to handle duplicate discovery and filtering
created a gui skeleton for the new duplicates page
started some handles and update code for the new duplicates page
wrote a new txt file in the db dir about the new emergency extract scripts
wrote an emergency extract script to migrate subscriptions to a new db
wrote an emergency extract script to migrate options to a new db
the trash clearing daemon now runs in the foreground, and a little of its code is improved
the trash clearing daemon now makes a popup message when it does work
the server's ssl keys are now set to read-only on Windows and user read-only only (i.e. chmod 400) on Linux and OS X on creation and update
added a explicitly unicode popup message to the debug test
fixed some network error catching code that was using Windows-only error codes
converted more of the thumbnail right-click menu over to the new system
improved some listctrl code
misc cleanup

version 239

finished up similar files search data maintenance code
similar files search data maintenance will now run during idle time
similar files search data maintenance can be called from database->maintain menu
the crowded database->maintenance menu is now split into maintain, regenerate, check
improved the similar files tree generation code, speeding searches significantly
wrote a new listctrl class to handle more complicated objects and also sort by underlying data
the new listctrl now handles object name non-duplication
cleaned a bunch of crap old listctrl code
manage export folders now uses the new listctrl
manage import folders now uses the new listctrl
manage subs now uses the new listctrl
manage scripts now uses the new listctrl
options media viewer options now uses the new listctrl
file import status panel now uses the new listctrl
the new listctrl can now quickly fetch item index from the underlying object
the file import status panel should now cope with extremely huge lists a bit better now!
multiple parsing child nodes can now import from/export to clipboard as lists
export folders now have names, so you can have multiple export folders pointing to the same path! existing export folders will get their path as their name, but this can be changed no prob
cleaned import/export folder dialogs
cleaned import/export folder dialog workflow
several misc import/export folder improvements
hydrus servers are now exclusively https with self-signed certificates
hydrus servers now create server.crt and server.key in their db folders for SSL--these files will be backed up along with everything else on an admin backup command
system:hash now ignores the file domain and any other predicate. it now returns very quickly, no matter the context
improved system:hash search logic
all the awkward choice dropdowns in system predicate panels are replaced with radioboxes
improved system:rating panel grid layout
wrote a better subclass of radiobox to handle more data
moved first half of thumbnail menu to new menu system
cleaned up a little thumbnail menu logic
improved the different ways services are added to thumbnail menu
thumbnail 'select' menu is logically cleaned up and allows for better file domain selection
the thumbnail menu's copy files and copy hashes to clipboard will now send them ordered (they were previously pseudo-random)
added 'paths' to the share->copy thumbnail menu for copying multiple files' paths. these are also ordered.
if the popup message manager does not have any errors, it will no longer unhide (which can annoyingly raise the main gui window) when the gui window does not have focus
removed some old redundant error reporting stuff in popups
improved and quietened some some mime detection failure code, sped up mime failure loop in all cases
massively simplified and atomised how new serialisable-object management panels can save their data
the manage subscription dialog now saves in a single, faster transaction
the manage script dialog now saves in a single, faster transaction
reduced redundant index work from analyze jobs
improved tag parse error handling
fixed media removal rules when deleting from the 'all local files' domain
polished and clarified some of the help's tag schema
misc cleanup
more misc cleanup
misc refactoring
more misc refactoring

version 238

added left/right equal ratio preference to the similar files search tree generation
added high standard deviation preference to the similar files search tree generation
added 'db profile mode' to the help menu, for briefer profiling in future
when a similar files search occurs with db report mode on, it'll popup a nice bit of text saying how many cycles it took
fixed a bug due to last week's 'all local files' change that meant archive/delete filtering would include any selected trashed files
'recent tags' will now only be populated by add/pend tag events, and only by those tags that were valid for that action
moved the help menu to the new menu system
added bitmapped menu item support to the new menu code
fixed and cleaned up some of the new menu code
improved a/c cache regen error reporting

version 237

added 'all local files' service that spans all local file domains
improved trash service code
trashed files now report their trashed timestamp in right-click menus
trash views will sort oldest/newest by the trash timestamp
renamed 'local files' to 'my files' to reduce initial confusion
if you don't like 'my files', you can now rename the local files service under manage services!
improved how some local file service metadata is stored
cleaned and possibly fixed up some delete code
cleaned up a bunch of misc service and file service code
the client is more intelligent about what local files are -- you can now 'open externally' trashed files, for instance
on multiple monitor systems, the new sizing system now bounds itself by the appropriate monitor's dimensions (previously, it was always consulting the primary, I think)
if an expansion event causes a frame to grow off screen, the new sizing system will now attempt to move it up and left so it is completely visible
fixed an important bug in the specific service autocomplete cache that was leading to several kinds of miscount--please regen your autocomplete cache at your convenience
the client can now post additional messages on boot (you'll see one!)
improved how errors with unusual characters are applied to failed import file status objects
left double-click on the main gui greyspace now opens the new page chooser as well
restored session pages now recover more gracefully from missing services
shortened the 'stop after this many files' phrase, which was mis-sizing downloader panels
changed several bits of old jank in the 'import options - files' collapsible. it is also now thinner
misc gui improvements
updated more of the menu system code
the selection tags box now sizes its height more reasonably
fixed 'check now' and 'reset cache' buttons on edit sub dialog
subscriptions report better file import status on their popup
misc cleanup

version 236

in prep for a network https upgrade, the client can now detect and escalate to https when making connections to hydrus services
import/export to png and clipboard now supports multiple objects at once!
rewrote the manage subscriptions dialog to work on the new panel system
the new manage subscriptions dialog has a listctrl and a sub edit dialog
the new manage subscriptions dialog has the same add/export/import/dupe/edit/delete buttons as the manage scripts dialog
subscriptions are now importable/exportable, including en masse with the new multiple object import/export support!
the new manage subscriptions dialog has retry failed/pause-resume/check now/reset buttons for easy mass subs management
the edit subscription panel has a bit of a layout makeover
the edit subscription panel now updates itself as its buttons are hit
the edit subscription panel disables buttons that are not applicable
subscriptions can now be renamed!
cleaned some misc subscription code
relabelled initial and periodic file limit in the subscription edit panel
middle-clicking on the main gui's greyspace (e.g. to the right of the notebook tabs) will spawn the new page chooser!
created a simple HydrusRatingArchive class--will do more with it in future
added ffmpeg, python, and sqlite versions to the help->about window
harmonised daemon code
added a new class of daemon that will not fire while a session load is occuring
subscriptions, import and export folders, and file repo downloads now use this new daemon
cleaned the way background daemons check for idle
expand/collapse panels now notify the new kind of toplevelwindow that a resize may be needed when they switch state
time deltas (like on subs edit panel or a thread watcher) now render more concisely ('7 days' instead of '7 days 0 hours')
serialisable object png export panel now has a width parameter
fixed a bug where tags that begin with unicode digits were accidentally identifying as numbers for the purposes of sorting and throwing errors on convert fail
the media viewer can handle some more unusual content update combinations--for instance, if it cannot figure out which media to show next, it will revert back to the first image rather than displaying an undefined null mess
updated and cleaned a bunch of my old misc encryption code
misc cleanup

version 235

finished first version of new faster dupe search--'system:similar to' now uses it
finished faster dupe search tree creation
finished faster dupe search tree search
added faster dupe search tree leaf insertion
added faster dupe search tree rebalancing maintenance
added faster dupe search tree orphan deletion maintenance
added faster dupe search leaf-regeneration-as-scheduled maintenance
misc faster dupe search tree work
new search tree will be created on db update
fixed the 'add' buttons on the import files dialog
fixed linux manage tags text input
reduced incidence of linux media viewer post-manage tags key event swallowing
fixed an issue where if the manage tags dialog was launched from the media viewer while a hover window was displaying, the main gui would often be raised when the manage tags frame was closed, annoyingly obscuring and de-focusing the underlying media viewer
fixed manage tags action workflow when no valid tag choices exist
entering a tag petition reason now has several common 'suggestions' for quick entry
removed ffmpeg binary from default linux release (as basically every linux has a decent ffmpeg by default)
fixed the incorrect ffmpeg version for os x
added an option to options->media to prefer to use the system ffmpeg over any in the hydrus bin directory
fixed linux arrow key navigation of the page picker dialog
cleaned up the page picker dialog's gross old code
the edit file lookup script panel now has a script management control on its test page to better report on and control tests
scripts will new correctly attempt to decode unicode and other encoded text in responses
if non-text data is accidentally fetched in the file lookup script panel's test page, it is now caught and a small summary of that event is printed to the text box instead
added the same non-text catch to the edit link node panel
improved how some script errors and status updates are handled so the new test window script management controls will see and report more useful information
removed 'restart' option from linux for now--looks like a bug in pyinstaller
improved how the new GET request read reattempt works
cleaned up some spammy error code
cleaned up spammy test code
misc error handling improvements
misc cleanup

version 234

the action choice workflow in the manage tags dialog now merges decisions for multiple tag entry events
the button dialog that pops up on the new merge manage tags workflow has button tooltips to better describe the proposed action's tag and file combinations
the manage tags dialog only requests one petition reason for an above merged multitag petition event
perceptual hashes are now stored in the cache db (moved from the preferably leaner main db)
the database now supports multiple perceptual hashes per file
fleshed out the perceptual hash vptree generation and maintenance code
added option for main gui title to options->gui--it even updates live
added unicode path support when importing serialised pngs
added unicode path support when exporting serialised pngs
exporting a serialised png now reports success via the export button, which will temporarily relabel itself
added discord links to help files and help->links menu
when a GET network connection fails during the read phase due to an unexpected timeout, the request will be reattempted a couple of times, like failed connection initialisations currently are
escape key now closes scrolledpanelframes (review services or import status frames)
manage tags dialog and frame now closes due to escape key correctly
fixed a size calculation bug that was not initially drawing scrollbars on manage options and any other listbook-containing scrolling panel when the screen is too small to show the whole dialog
undo menu now works on the new menu system
cleaned up some bad gui-thread interaction in the file import dialog
fixed file->restart in the built release, including when the install path includes a space
the trash service no longer records which files it has physically deleted, as this information is not used and is redundant compared to the existing local files' deleted record (existing records will be deleted on update)
the subscription daemon will wait 90 seconds after boot before triggering--quitting the client before then will result in subs not being checked

version 233

made a plan for faster dupe search
created skeleton of db tables for faster dupe search
wrote out search algorithm for faster dupe search when the rest is ready
updated search algorithm and skeleton to support a future multiframe (i.e. gif/video) similar files comparison
the os x release now has 'client' as the main client executable
the os x release now includes the server, under the 'server' executable
added studio (128,0,0) and meta (0,0,0) default namespace colours
the 'add all' button on the file lookup tag suggestions panel now will only ever add--it won't remove/rescind pend
the password system now supports non-ascii input (be careful though, as even a subtle change in keyboard encoding that nonetheless may have the same visual characters will likely be considered a different password)
fixed the 'nonetype has no dtype' rendering problem introduced by last week's 16-bit channel fix (this mostly affected static gifs)
fixed non-null ratings changes in the numerical ratings dialog
fixed culling and adding variables not initialising on the edit html formula panel
fixed the htmlparser attribute fetcher to deal with both single value attrs ('id') and multiple value attrs ('class'). in the latter case, the many values will be joined up, as how they appear in html
tweaked disk cache timings a little more to account for more scenarios--it now also reports itself to the shutdown splash screen

version 232

finished png object sharing system, created some gui for it
switched the top half of the new png export to wx code, which can render with prettier system fonts
you can now 'export' parsing scripts to png from manage parsing scripts for easy sharing
you can also 'import' parsing script pngs in the same dialog!
the file lookup tag suggestions panel now listens for media update events (so if you browse the media viewer with manage tags open, it'll keep up with the current file)
added a 'add all' button to quickly add all tags from file lookup suggested tags panel
created a 'script management' control for file lookup suggested tags panel and added some script text status updates for it (progress gauge doesn't do anything yet)
added basic 'cancel' support to script management control--mid-http cancel support will come with the gauges
added url support to thread notification objects, and hooked script code into it
added a url menu button to script management control, so you can launch any parsed urls the script found in your browser
html parsing formulas now support positive and negative front and end character culling
html parsing formulas now support prepending and appending arbitrary text
added some more pretty text to the html formula summary to explain when they do the new culling or adding
script 404s and other network errors are now caught neatly and reported
link node 404s and other network errors are now reported and recovered from
fixed favourite file lookup script selection on initialisation
fixed parsing veto test to include membership of search string in result, not just exact match
fixed html formula tag attribute fetching
updated os x release to python 2.7.10
fixed os x about window
fixed v231 linux release image rendering crash
updated linux release to wx 3.0.2.0
updated linux release to opencv 3.1.0
fixed rendering for 16-bit-per-channel images
fixed a potential out-of-order settagboxfocus event issue on panel initialisation
created a 'menubutton' control and switched existing code over
created some 'nullipotent panel' dialog code to handle some new stuff
misc fixes
bunch of misc refactoring

version 231

added file lookup scripts suggested tags control and appropriate options panel
added final bit of gui->filelookup script->content update pipeline conversion and tie-in code
moved all the suggested tag options into a notebook
added notebook layout for suggested tag control columns. it is now the default
suggested tag columns now have a unified set width
the parsing ui's add script and node buttons now spawn menus rather than the awkward listofstrings dialog
added 'iqdb danbooru' file lookup script. it hangs the ui pretty bad atm, but that will change in future
fixed a critical pyinstaller problem with the os x builds
log files will now be appending with 'year-month', and will roll over to a new file as the month turns
help->about dialog now has some library version information
converted more menu stuff to the new system
wrote the guts of a new png-based object sharing system
dropdown choices are more resistant to missing init and invalid defaults
tweaked some disk cache maintenance timing
fixed a multi-version update bug regarding external thumbnail paths not being initalised
misc cleanup
misc cleanup, improvements

version 230

all multiline text ctrls now support ctrl+a for select all, wew
added dynamic menu_item tracking to new menu id system
added submenu tracking to new menu id system
added comprehensive menu, submenu, and menu item destruction to new menu id system.
'pending' menu now works on the new menu id system--is likely a major cause of the menu id bugs
added 'info' tabs to the script/nodes/formula parsing gui panels
added a 'veto' content type for discovering undesired redirects
added special veto support to parsing engine
expanded the content node panel to handle and display multiple content types
added 'link' parsing node panel
expanded 'link' panel test page and tied in secondary layer example url and data forwarding to children
fixed the parsing node children control not forwarding example data up to nodes on edit
file lookup url is now explicitly stored in the file lookup script object
file lookup url is now passed to child parsing dialogs for test example data purposes
expanded internal 'requests' interface a little--it now also always includes the 'user-agent' request header as the typical 'hydrus/version_number'
wrote script GET/POST engine gubbins and tied it into the respective test page
wrote link GET engine gubbins and tied it into the respective test page
'person' namespace added to default namespace colours--it is RGB ( 0, 128, 0 ), a darker version of 'character' green

version 229

edit html formula dialog can now be launched from edit content node panel
edit html tag rule dialog can now be launched from edit formula panel
edit formula panel test now works
edit content node panel test now works
edit script panel test now works
html formula now filters out missing content rather than complaining
misc parsing work
fixed some listctrl resize columns to be more helpful
attempting to size a resizing listctrl column will now move the resizing column to the last column, which will avoid the flickering scrollbar sperg-out of previous
attempts to resize the final column of a listctrl while it is the resizing column will be silently vetoed, which will avoid more resize event flickering
frame location options panel now allows for negative position coordinates (for multi-monitor setups)
all the new resizing panels now handle certain size updates in a more unified and reliable way
fixed an issue with hover windows' mouse position check breaking during media window init/shutdown edge cases
avc1 mp4 support added
ftypFACE mp4 support added
ftypdash mp4 support added
mpegts mpeg support added
to help migration, paths are now only stored as portable (i.e. relative to the base path) if they are beneath the base path (previously, this was true for any path on the same partition, permitting PITA '../../my_files' portable paths)
to help migration, portable paths are now relative to db_dir, not installation base_dir (existing client_files_locations, ideal client_files/thumbs_locations, export_path, and HTA paths will be updated)
started work on cleaning up button code across the program
created a better way to bind menu events that shouldn't suffer the id overflow bug
started work on replacing the old id_to_action_cache menu mess with the new system
too-many redirect exceptions are now richer and caught more sensibly
circular redirects are caught as soon as they occur
reduced sqlite cache size (with the new disk cache, this is now less important)
client.py is now included in the source and windows frozen releases

version 228

added support for RIFF .avi files
fleshed out parsing ui, made progress on node editing, moved edit/test panels to notebook pages to reduce clutter
the video renderer now initialises off the gui thread, which should reduce some video browsing chunkiness
trying to save a session that would overwrite another (from the same name) now throws up a yes/no warning
reintroduced '--no_daemons' and '--no_wal' to the new command line argument parser
you can now set the default tag service for new search pages under the file->options->tags page
since it can now be anywhere, added 'db directory' to the file->open menu
fixed a build capitalisation issue that was making the windows frozen exe release crash when trying to run from source
the 'too many redirects' error message is expanded to print the url called and its redirect request
errors from trying to delete files in use by another process will no longer make popups--they'll just write a note in the log
improved how some errors are written to the log
misc fixes

version 227

both the client and server now use standard command line input and will produce a proper help with a -h switch (although the 'windowed' frozen client executable can't print back to console)
both the client and server now support a -d or --db_dir to set a db directory outside of the install directory. you can now run multiple clients off the same install!
critical boot error reporting is more reliable
if the admin port is in use on server boot, the error is handled better and the program will quit more easily
fixed mime parsing for some unusual webms
added support for even more unusual webms
added support for SonyPSP mp4s
fixed system:hash when searching for a non-existent non-sha256 hash
fixed layout of the 'dismiss all' popup button
preview windows will no longer update in the background if the 'hide preview window' option is set
misc fixes
misc cleanup

version 226

fleshed out a bunch of parsing script engine and gui framework and added an 'under construction' entry to the services menu. feel free to play around with it!
fixed autocomplete same-service sibling count merging
sped up how autocomplete counts are put together
results will build faster due to an improvement in how tag rows are fetched
results will build faster due to an improvement in how tag strings are fetched
incidence taglist sorting now secondary sorts by lexicographic in the 'correct' direction (a-z for desc, z-a for asc)
all listctrls now support ctrl+a to select all
cleaned up some superfluous and possibly buggy canvas zoom calc when using the open externally button
the zoom_switch event will now reset the drag coordinates (recentering the image) if the resultant zoom fits into the canvas frame
the mouse cursor hides itself more quickly on media drag on Windows (so warppointer jitter is hidden)
the popup message manager will be careful about switching from hide to show while the parent gui is minimised on windows, which should reduce the grey box problem
the popup message manager will specifically re-layout on a transition from hidden to shown to stop the 'crushed into a corner' layout problem that would sometimes happen here
popup message manager will now consider a minimsed gui as unfocused for the purposes the of the focus BUGFIX option
added a new delayed popup message item to the debug menu for testing minimised popup creation
vacuum maintenance period option is moved, reset to larger default of 30 days
'last session' will now always be updated on client close, even if it is not the default session

version 225

system:numtags is now much faster when applied to 'local files' or a file repository
system:numtags now correctly counts same-tag-different-namespace tags as distinct (for instance, [page:1, chapter:1] was previously being counted as only one)
refactored fast db integer iterable access code into a context manager
media result building is faster
added namespace-grouped incidental tag sorting
fixed the OpenCV image loader for monochrome images
fixed a thread interaction issue when drawing popup messages
popup messages should no longer flicker while static
popup debug test is richer
fixed editing of some taglists such as in explicit tags and import tag editing
SSL EOF errors are now caught by the networking engine

version 224

rewrote the static image rendering and caching pipeline -- images are now resized on the fly, and only the master image is cached
all image rendering and zooming is faster and consumes less memory
image rendering now obeys the zoom quality options in the 'media' options page!
static image zooming will take advantage of OpenCL (video card acceleration) wherever available
added an option to the 'media' panel to allow fast but potentially unstable opencv image loading
fixed the tumblr gallery downloader (tumblr put an extra character in their API response, wew)
the debug code profiler now prints more information
cleaned and updated reducing lag help page to match new profiler
'recent' suggested tags are now sorted by recency
the thread watcher now checks url history as well as md5 match to compensate for cloudflare optimisation making for unreliable api and causing dupe downloads
misc fixes

version 223

fixed the popup message manager's bad minimise recovery
added an option to automatically 'hide' the popup message manager on main gui minimise (which has patchy multiplat support--some mindow managers don't do this automatically, some do but break when you force it)
added an option to automatically hide the popup message manager on main gui dofocus, which may help with window managers that minimise to the system tray, which wx cannot detect
added a 'make some popups' test job to the help->debug menu
siblings will now present in a more service-specific way
many gui elements that display tags are now aware of their tag service for the purpose of collapsing siblings
simplified a whole load of the siblings code
in some conditions, media will load faster
autocomplete tag censorship will now also apply on in a service-specific way in 'all known tags' queries
another remote connection reset exception is now properly caught by the network engine
the embed button will now draw the correct background colour behind transparent thumbnails
the animation scanbar will no longer scan if it has not previously experienced a mouse_down event--it could sometimes inherit this status from a previous media filter or embed reveal, resulting in undesired instant scan and undefined animation canvas behaviour
advanced content update panel will no longer list 'copy' as an action if 'local tags' is the only tag service
fixed a bug when performing a 'go!' action in the advanced content update panel when 'local tags' is the only tag service

version 222

created a 'raw url' downloader page that just downloads urls and tries to import the result. it has a 'paste urls' button to make mass import of a list of urls easy
fixed an options update bug when updating to v221 from any version before v220
added support for 'ftypqt' quicktime (usually .mov) video
embed button now uses system gui colours
embed button now puts the thumbnail of the media, if one exists, behind the 'play' button
sped up an inefficient existing mapping check that was slowing new pending mappings for popular tags
'last session' will no longer be listed on the gui session delete menu
cleaned up the main gui's initialisation events--a sizing bug often triggered after system reboot may be fixed
popup messages are initialised in a safer way
popup messages are dismissed in a safer way
popup messages will hide/restore themselves more reliably when the main gui window is minimised/restored
the pending popup message queue is now regularly purged of already deleted messages
new popup messages will no longer raise the main gui window to the top
subscription http errors during the gallery sync phase are now caught and handled gracefully, with exact error text written quietly to the log
network timeouts during successful response read are caught and converted to a hydrus network exception that will be caught and handled more reliably up the chain
the client's upnp daemon will now silence upnp mapping errors that are due to the router being too busy or full or any other unknown errors. a simple statement about the error and an instruction to explore the problem with the manual upnp manager will be written to the log
finished flexgrid refactoring
the new automatic flexgrid creation detects subsizers and lines them up more accurately with standard controls
wrapped the different sections of the 'speed and memory' options panel into staticboxes
wrapped the misc crap up top the 'tags' options panel into a neater staticbox
taglists with unusual tags will copy them more reliably and present fewer invalid menu options
misc layout fixes

version 221

fixed the 8chan thread parser for the new sha256 file urls. legacy links should still work!
created a 'x recent tags' suggested tags control and all the db and so on to go with it
recent tags defaults to off--turn it on under options->tags
replacing the previously basic zoom_in_to_canvas, the zoom options now support 100%, max regular zoom, or canvas fit for scaling up and down and for the media and preview windows!
cleaned up some garbage zoom code
refactored and harmonised how default zooms are calculated
harmonised and improved how canvas zooms are set
zoom_switch now works for flash
if it looks like the last instance of the client did not shut down cleanly, a new client will now present a dialog to choose whether to attempt opening the default session or just a blank page.
all listctrls across the program have better secondary sort
added an isatty() stub to the HydrusLogger for any unusual error handlers wanting to call it (youtube parser pafy was doing this)
removed some accidental (123) counts from some predicate menu and clipboard rendering
refactored a bunch of redundant flexgrid creation code
did a bunch more flexgrid refactoring
refactored some ugly global variables
misc mvc cleanup
misc cleanup

version 220

fixed collection selection show action lookup for the preview window
files added through the normal import dialog or import folders will be not be fed into the import pipeline if it looks like they are already in use by another process
(hence setting an import folder to a browser download destination should be more reliable)
rewrote some critical phash generation code that was running extremely slow when importing large pngs with transparency
all listctrls that have an edit button now also trigger the edit event on item activate (e.g. double-click/enter key on an item)
added zoom in/out/switch to edit shortcut action dropdowns, can't promise it works everywhere
'stopping' a gallery page search will now correctly make sure the search queue is unpaused so it can move on to any subsequent search
the way excess messages are added to the popup message manager is quicker, improving gui responsivity on message spam
fixed a potential race condition when a file import occurs at the same time as client_files rebalancing
fixed a potential race condition when a check_file_integrity db maintenance occurs at the same time as client_files rebalancing
the client will no longer suppress some core debug stuff
fixed an invalid event handler on canvas close bug
fixed a long-time wx locale issue
fixed a misc main gui parent assert issue
fixed some invalid non-wx-thread calllater calls
cleaned up some inartful showmessage calls
cleaned up some datacache init code
fixed some shutdown event handling
generally improved and fixed in some cases how threads signal job status
improved some thread-interaction timings
improved how files and thumbnails are deleted
file imports add their files to the client_files structure in a more sensible way
misc fixes
misc layout fixes
misc cleanup

version 219

wrote framework for per-mime zoom options--mimes now have separate show actions for the media viewer and the preview window and zoom in to fit
added 'half/double zoom' to limit a mime's zoom to only 25%, 50%, 100%, 200%, 400% and so on (png defaults to this)
added placeholders for zoom in/out algorithm quality, but they don't do anything yet
added a listctrl to edit all this to options->media
added a scrappy zoom values option to options->media
cleaned up a bunch of zoom code
added video/mpeg support
fixed some mime layout stuff
duplicate tag predicates across the 'all known tags' space are merged into more accurate (3-5)-type counts
refactored and simplified how predicate counts are merged
if launched from the manage tags dialog, the advanced content update dialog now correctly filters hta imports to only the files the manage tags dialog was launched with
the advanced content update dialog has a bit of better text to explain this
in prep for a complete rewrite of the image rendering pipeline, merged the fullscreen and preview image caches and cleaned up some related code
added resume recovery to the v215->v216 update code for users for whom this update was interrupted
directory creation is improved across the program
fixed the optimised merge-move file code, which was forgetting to clean up the source when no move was needed
the optmised merge-move directory code will now be much quicker in some situations
all file moves across the program should be less stupid-error prone, and repeating/resuming many maintenance or update tasks that require a lot of moves will be much faster
manual exports, which use copy rather than move, should also be more sensible and faster when repeated
improved database exception rendering, which in some cases was being truncated
misc cleanup
misc small updates to help

version 218

related tags control now updates on new media events (i.e. if the parent media viewer scrolls to different media)
related tags also initialises with suggestions and refreshes the list on media change
related tags now properly filter out dupe suggestions caused by siblings
went over all the gui colours in the client, cleared some redundancies and replaced all the static whites and blacks with appropriate system defaults or chosen custom colours
full size thumbnails can now also be stored in a different location, like resized thumbs, under options->file storage locations
fixed false positive shutdown maintenance checking due to vacuums projected to take longer than the given maintenance time
fixed false positive shutdown maintenance checking due to sqlite_stat1 creeping in
pasting tags or otherwise editing many in at once in the manage tags frame will immediately commit all the changes in one db transaction, rather than one for each tag laggily in turn
removed 'import tags' from file menu--this advanced stuff is now done in and better left to the advanced content update dialog
restoring from an old db backup with a simpler db structure will no longer create conflicts
deleting currently rendering files from the trash should be slightly more reliable
fixed some crash error reporting
misc cleanup

version 217

fixed some high-res video streaming thread scheduling problems with the new video renderer
fixed a cause of huge memory bloat with greatly upscaled videos
to improve seek response time, streaming buffer for the video renderer has a much smaller cap
renderer throttling calculations are more sensible and reliable
the video renderer discards frames to save time if they happen to still be in its buffer
the video scanbar now displays the current frame buffer around the caret!
video canvas now recycles the same frame blit bitmap to save a little time
wrote a prototype related-tags suggestion 'service' for the suggested tags control
you can turn it on and set some options for it at options->tags, feedback would be appreciated
munged increasingly complicated components of the suggested tags control into a clean and proper self-hiding panel
fixed a very important bug that was failing to filter visible thumbnail fetch on mass select and thus massively slowing down the client on large ctrl+a-like operations
open externally button now shows the media's thumbnail, if it has one
open externally and embed buttons now use hand cursor
the simple path tagging dialog panel now cuts off .jpg extensions from filenames on filename parse
if the string component of a generated file export path already ends in the correct .jpg extension, a second will not be added
ipfs unpin will no longer break if the file was already unpinned
the hydrus server now gives filename (for a file save as dialog) correctly on a content-disposition header (this affects the client's local booru as well)
the secondary sort can now be a namespace or rating sort
fixed some potential init problems with some dropdown controls
an edge case object-missing cache retrieval bug is fixed
updated openssl on os x, which might have fixed some problems
updated python on windows, which updated openssl and a bunch of other stuff
updated sqlite on windows
updated linux dev machine to ubuntu 16.04, so a variety of packaged libraries are updated
fixed auto server setup if the client is launched from a windows cmd window
misc cleanup

version 216

video rendering pipeline rewritten to be much smoother
canvas video render timing rewritten to be more frame accurate and smoother
video rendering pipeline will deal with 100% CPU rendering bottlenecks much better
videocontainers will rush to render towards clicked cursor and then slow down
videocontainers will stop rendering as soon as they are replaced on zoom
videocontainers will stop rendering as soon as their parent canvas is closed
added a video buffer size option to options->speed and memory
client_files subfolders are now split into f, t, and r groups for files, full size thumbnails, and resized thumbnails
added an option to override resized thumbnail storage location under options->file storage locations
added secondary sort option to options->sort/collect
fixed a bug where automatically pended tags could exist on top of current tags, forming redundant (1) (+1) situations
fixed a bug when sorting by namespace and a respective namespaced tag has a sibling that de-namespaces it
relatedly, namespace sorting will now filter correctly by collapsed siblings when those sibling pairs' namespaces differ
and collecting will now collect correctly with sibling pairs where the pairs' namespaces differ
simplified how the server caches its account sessions
improved how the server refreshes specific account sessions
the server now refreshes cached session accounts immediately when an admin actions an account type
neatened a little server db account fetching code
fixed file repository petition denial
added a better catch for upnp addmapping error when the mapping already exists as a port forward
the media canvas will no longer accept any mouse or key event if a menu is open anywhere in the program (this circumstance is difficult to pull off, but does cause a crash)

version 215

you can now set different suggested favourite tags for different tag services
added an option to tags option panel to set fixed suggested tags control width
file import status now works on the new window sizing system, and when launched from main gui uses file_import_status frame key
file import status is now a generic apply/cancel dialog when launched from import folder or subscription dialogs
these file import status dialogs are reenabled for linux and os x, as they work properly now!
very large images will render faster and more beautifully on a freshly launched fullscreen media window
all zooming in will be higher quality, particularly for sharper images
better zooming in code also works for very large animations/video
started a framework for better html parsing
updated sankaku downloader to support 'medium' namespace
analyze calls no longer analyze sqlite_stat1, which isn't needed and sometimes gives awkward maintenance timings
fixed server's deleteorphans, which wasn't working with the new thumbnail names
improved speed and weight of server's deleteorphans
fixed a missing foreign key in file_petitions
improved thread worker creation and job processing scheduling
fixed os x media viewer window float_on_parent status
default gui session now defaults to 'last session'
gui recovers from missing default session on boot more gracefully
crash log code improvements now applied to all boot scripts
misc cleanup

version 214

wrote a new resizing dialog 'edit' class for simple resizing dialogs
added comprehensive frame size and position options for the new system to the gui options panel
moved review services frame to the new sizing system
windows that initialise maximised will correctly return to their last remembered size and position on a restore event
maximising a window by dragging it to the top edge of the screen should remember last position as the initial drag start position more reliably
positioning code is a little safer
fixed some missing recalculation of best/min size for wx.notebooks after page change
fixed missing recalculation of scrolledpanel's virtualsize after child wx.notebook's page change
fixed bad parentage for file import status frames
hid file import status button in manage import folders and subscriptions dialogs for non-Windows, as this is very broken, and the parentage fix wasn't enough
improved fuzzy padding on size calculations
refactored and cleaned and harmonised a bunch of the new window resizing code
suggested tags - favourites tag entry in options is now a live autocomplete dropdown
cleaned suggested tags - favourites layout in options
suggested tags listbox now sets its width to exactly fit its tags
improved workflow logic of removing/petitioning siblings and parents (shouldn't get stuck in loops as much now)
tag listboxes will update when tag siblings change
the 'auto-replace siblings' state on manage tags will no longer incorrectly apply to removal actions
import status caches now display errors in a more straightforward way
errors sent to import status caches are now also printed to the log
simplified how database exceptions are caught and displayed
database exceptions now preserve the original exception type
fixed db-side traceback line spacing in database exceptions
improved general database exception rendering
fixed imports for videos with negative start time
deleting videos from the trash that are currently rendering should be more reliable
crash.log now goes to the db dir, unless that isn't determined yet or is unwritable, in which case the traceback goes to console

version 213

created a new tag listbox for the new suggested tags control
added a extremely basic and prototype suggested tags control for favourite tags to options->tags and manage tags dialog
off-screen position detection is now more lenient to account for window managers that position slightly off screen to cut off border
new dialog sizing code adds a little more fuzzy padding on all min size calcs, reducing superfluous scrollbars
hydrus's custom listbook now informs parent scrollable panels of need to recalc scrollbars on page change
the new flexible dialog and frame now catch these events and will expand their size if they need to and have the space
the client options dialog now works on the new sizing system--it will automatically resize when you change pages
some frame resize save event code is improved
the media viewer is no longer aggressive about claiming focus
stacked manage tags panel buttons to allow for thinner dialog
misc dialog cleanup
corrected some bad logic that was excluding transparent gifs from the new thumbnail handling
all gifs and pngs will have their thumbnails regenerated on both the client and server, taking advantage of the recent thumbnail improvements
fixed a bug if you click cancel when choosing an hta path to sync a tag service with on manage tags dialog
fixed a bug where noneditable services (local tags and local booru) could sometimes become editable on manage services dialog. the edit buttons should now disable correctly on init, and there's an additional check in the add code in case of future problems
any superfluous local files/booru services will be deleted on update

version 212

wrote a neat flexible system for recording and restoring a variety of window size and position information
moved the manage tags dialog to the new size and position system
moved the main gui and the media viewer to the new size and position system
the manage tags window launched from a media viewer is now a non-modal frame, allowing interation with the underlying media viewer while floating on top of it
the new manage tags frame commits changes immediately!
review services and file import status will now stay on top of the main gui
fixed some bad advanced content update dialog launch code
the hydrus tag archives in hta syncs can now be anywhere, not just in the client_archives folder
whole bunch of hta sync cleanup
paletted images are now dequantized to RGB(A) before thumbnail generation, enhancing scale quality
'LA' (greyscale+transparency) images will now render with correct transparency
png files will now always generate png thumbnails (previously it was just ones with transparency)
trivial options changes are less frequently saved
rule34@booru.org now parses namespaced tags
fixed a bug in thumbnail resize error recovery code
fixed a bug in file repository petitions
updated to Pillow 3.2.0
misc cleanup

version 211

added options for disk cache init and maintenance to 'speed and memory' page of options
abstracted the manage tags dialog into a general purpose 'apply' dialog and a scrollable manage tags panel
the 'auto-replace entered siblings' checkbox now remembers how it was last set
uncertain sibling tag counts will now appear like 'tag (1,234-1,456)', showing the range of possible results they represent
fixed a bug where siblings would be collapsed inside the tag manager object if a specific tag search domain was set
altering the search text on an autocomplete from 'tag' to '-tag' or vice versa should graphically update more reliably, and an exact-matching predicate will be promoted to the top of the list correctly either way
calls to upnpc will now kill the process and raise an exception if it hangs for more than thirty seconds
misc improvements and cleanup

version 210

manage tags dialog now launches much faster when thousands of thumbs are selected
manage tags dialog now has a prototype 'do this for all' checkbox to make mass selected-tag editing a bit quicker
polished the new disk cache turbo mode, added free memory limits and more options
a bit of disk cache population now runs on client boot, just to kick things off
a full disk cache population now runs before repo sync
the client now keeps about 200MB of itself in the disk cache during idle maintenance
the update files in client_updates are now stored compressed
the log files and 'already running' files are now stored in the db folder, meaning hydrus shouldn't have to write to anything but db dir and temp dir
'already running' files now delete themselves on program close if they can
the log directory is no longer needed
if there is only one booru to select, it will be auto-selected
client failed exit is less likely to hang, more likely to log error
client won't bother to save lest session regularly or on exit if this is not the default startup session
hotfixed an auto-vacuum bug
added a catch for any future vacuum mess-ups to save the db from collapse
added other catches for other close/reinit db calls
misc cleanup

version 209

the advanced content update dialog can now be !aunched from manage tags, and will only affect the files the dialog was launched for
gave the advanced content update dialog a bit of a makeover
system:ratio now supports 'taller than' and 'wider than' operators
merged client thumbnails into the dynamic file directory structure
the client files manager now manages thumbnail paths exclusively
thumbnail import responsibility moved to the client files manager
thumbnail resizing responsibility moved to the client files manager
orphan clearing and thumbnail regeneration maintenance tasks moved to client files manager
'regenerate thumbnails' now has an 'only do missing' mode
thumbnail regen on resize is faster
copy file to clipboard responsibility moved to the client files manager
media file and thumbnail deletion responsibility moved to the client files manager
if a full size thumbnail is missing but regenerated ok, the client files manager will now only produce one, better error message
cleaned some bad error detection and reporting code
cleaned some regular 'thumbnail size has changed' code
fixed and cleaned a mix of file and thumbnail path code
client files rebalancing and recovery will now efficiently move-merge rather than copy-merge
fixed some bad portable->absolute path normalisation
improved missing external storage location error reporting
server thumbnails are similarly merged into server_files
to keep things simple, the client no longer keeps track of number of local vs. remote thumbnails for file repositories
perceptual hashes are now calculated from original file, not thumbnail, and only for jpegs and pngs (videos had too many black frames etc...)
the vacuum maintenance popup will now only appear if a vacuum can be completed in the time available
slideshows will now yield to any open popup menu, not just those launched from their own canvas
wrote a prototype turbo mode and put it on the debug menu
misc cleanup

version 208

split the path tagging dialog's complex widgets into simple/advanced panels
added filename and directory 1, 2, and 3 easy tagging, with namespaces to path tagging dialog
fixed up some buggy 'tags just for selected files' behaviour
ipfs directories now support a note
the ipfs directory note can be edited from review services
added 'all namespaced tags' and 'all unnamespaced tags' to the advanced content update/service-wide operation dialog
added 'clear trash' button to trash panel on review services
separate timestamps for file repo upload and ipfs pin will be displayed on single-thumb right-click menu
the correct file domain timestamp will be used for newest/oldest sort
db integrity check now scans all attached database files
broke up mappings petitions in the server db into more efficient 'deleted' and 'petitioned' parts
cleared out some static refs in prep for external db location
simplified a bunch of file mirroring code and improved error handling of directory mirroring
fixed a rare thumbnail resolution calculation bug
misc smaller fixes and cleanup

version 207

finished ipfs directory creation
ipfs review services pane now shows current directory shares
cleaned up some content event responsibility in ipfs service code
fixed a file download bug when using ipfs 0.4.2
wrote some more ipfs help, for directories
popup messages can now hold arbitrary copy-to-clipboard text content
fixed the new listbook changes for Linux, which was having trouble tracking the client_data inside the sorted listbox
fixed an issue with the custom filter management dialog possibly duping the 'default' entry on ok, making impossible to again launch
offloaded tag parent generation responsibility in the a/c write tag pipeline
added a checkbox to manage tags dialog that will control if siblings are auto-replaced on entry
all the clunky 'add preferred sibling' workflow is removed from the manage tags dialog
fixed incidence tag sort order not initialising properly in the options dialog (and transferring to lexi (namespace) every time)
fixed too-eager quit during shutdown db analyze
fixed the db restorebackup call, which wasn't closing the gui correctly
the manage services dialog will now automatically re-check 'allow zero' on numerical ratings with only one star
misc cleanup

version 206

if the currently focused thumbnail is removed from view, the earliest non-selected thumbnail will be remembered as a 'ghost' focus if the user then presses the arrow keys to continue to navigate
if the user presses an arrow key to navigate the thumbs when there is no known focus thumb, the first will be assumed
the listbook now supports duplicate display names
services of the same type can now have the same name
cleaned some listbook code
inbox or custom filters will now render first-file animations correctly on initialisation
fixed the 'change' event on the new timedeltabutton, which was not updating the thread checker on dialog ok
subscriptions will now naturally terminate their gallery sync on a 404, rather than dumping out with an error
fixed some false positive 'paths are different' testing in default directory mirroring code (used in db backup, restore) that was slowing these operations down
if database files have the same file and modified timestamp, they won't be re-copied in a backup or restore
client external storage location recovery/rebalancing will now skip over files with the same size and modified date
if you backup the client and have external file storage locations, a popup will remind you to back those locations up manually
did a quick pass over some help stuff, and added a 'can the client manage files from their origial locations' bit to the faq
created a simple no-reward patreon, wrote some help about it, and added links in the usual places
empty 'read' a/c dropdowns will be more careful about idly refreshing their system preds, hopefully leading to less accidental gui hang on large jobs
main gui menus will be more careful about refreshing themselves when called, hopefully leading to less accidental gui hang on large jobs
text snippets are stored inside the client db in a better way
added flat service directory structure storage support to the client db
fixed a mapping petition rescind sql typo
fixed the v198->v199 server update code
misc cleanup

version 205

fixed v201->v202 update code, which v204 retroactively broke
wrote a new maintenance routine to regenerate the ac cache if it ends up miscounting or not generating correctly
fixed specific ac_cache generation code, which was double-counting because of recent file-add optimisation
client db maintenance now vacuums attached databases according to their own vacuum clocks
client will more aggressively vacuum and analyze if the client has been idle for an hour
client vacuum now prints success statements, with duration, to the log
analyze now produces an auto-dismissing vacuum-like popup message
the 'is maintenance due' check on shutdown now accounts for the new vacuum logic and includes analyze needs as well
shutdown maintenance will use more of its allotted time if needed
tag archive sync now always filters to local files on initial sync
if you import a tag archive with sha256 hash, you are now given the option to filter to local files or import everything
the occasionally crashtastic opencv.imread call is now completely replaced with PIL, let's see if it's too slow
the new-but-now-redundant 'disable opencv for static images' option is removed as a result
fixed a redirection location parsing bug that was affecting sankaku subs with >1000 initial files
fixed some bad analyze timestamp tracking code
fixed advanced content update code, which still had some mapping service_ids floating around

version 204

current, deleted, pending, and petitioned mappings are now stored on service-separated dynamic tables
any tag service deletion or reset should now only take a few seconds
fixed a physical file deletion bug that seems to have affected (some?) versions of linux and os x
reintroduced a revamped 'clear orphans' database->maintenance routine for the client, which will delete or move orphaned files and delete orphaned thumbnails. run this if you are on os x or linux
improved misc file deletion code
fixed some multiplatform path conversion, and generally improved how paths are stored and retrieved
the network application of a parent tag to all files with the child tag now happens serverside, and only on petitioner-approval of a tag parent relationship. it still occurs locally to the uploader, but is now wrapped up in the parent commit
fixed some numtags search logic
fixed an issue when mixing system:age with any tag-based predicate
media_results db calls no longer require a file domain
'hide inbox and archive preds if either has no files' now defaults to off
added an option to disable OpenCV for gif rendering
the selection tags panel now initialises with the correct tag domain (it was previously always starting as 'all known tags', which is not always true for session pages)

version 203

thumbnail resize now happens on the fly--feel free to change it as often as you like, it takes no time at all
added 'lexicographic (grouped by namespace)' tag sorting to the selection tags box
added an option to disable OpenCV for static images under the media options page
panning with the shortcut keys now pans by a twelfth of the media size or canvas size, whichever is smaller
cleared the analyze timestamp cache for both client and server, which will force a reanalyze of the new db files on the next db maintenance run
improved some misc analyze code--primary keys are analyzed again
added an explicit analyze command to the client's database maintenance menu
the server now analyzes attached database files
improved an unhelpful server thumbnail error message
added an invalid NULL check-and-skip to hash cross-referencing
fixed some invalid a/c write dropdown search domain initialisation
fixed some borked zoom calculation code that was sometimes lagging the media viewer and leading to 100% zoomed images being sent unneccessarily and then being nastily scaled down
fixed a file import error if a synced tag archive is missing
updated sqlite for windows
misc cleanup

version 202

fixed a problem with the v198->v199 update step
added a bad error catch in the vacuum step in v200->v201
added some rollback exception handling for unusual situations
the autocomplete caches under the client_cache subdirectory are dynamically folded into a single client.caches.db file, and that file is folded into the main db journal like the other attached dbs
ac_cache file and tag processing lag and edge-case autocomplete miscounting should be removed as a result
cache generation is optimised for empty services
specific file_caches' add and delete file commands now modify their tables directly, saving a whole load of time that was previously done superfluously filtering
the specific ac_caches' add mappings command is a bit more logically efficient
increased cache size for each attached database
folded all 'attached' db update code into the main update_db routine
folded all 'attached' db init code into the main create_db routine
made the vacuum checking routine safer
reduced analyze log spam
analyze maintenance breaks its larger jobs up better
analyze mainenance now analyzes external databases
the db connection now refreshes every half hour, to regularly clear out journal files
the hydrus db assumes it generally has exclusive control over its db, so it no longer wraps its read-only requests in transactions
session load now happens off the main gui thread, and media generation is broken into 256-file chunks, which reduces startup db and gui lag for large sessions
the youtube video downloader now lists a webm option if youtube provides one (they seem to offer 360p webms on everything(?) now)
the timedeltacontrol (where you set the period for the thread checker and import folders and so on) is replaced by the more compact timedeltabutton, which will launch a dialog with the old control when clicked
the server will no longer revisit old updates and cull since-deleted content, as this was proving cpu-burdensome and not helpful to future plans for update dissemination, which will rely on static update files
misc cleanup of some server db stuff
content update processing will not spam menu updates throughout, but only notify once at the end, which should reduce idle gui hang due to db access choke
the autocomplete tag entry will not refresh system pred menu during idle, which should reduce idle gui hang due to db access choke
shutdown repo sync will now report the update summary text and will not spam the intervening statuses to the log
moved timestamp out of the serviceless media_result and into the locations_manager, to reflect the database's knowledge of each current service having a different timestamp
local and trash timestamps are generated from the locations_manager now, and non-local file service timestamps will soon follow (e.g. 'pinned to ipfs_service 3 days ago')
if the timestamp is unknown for the current service context, it will not be displayed
file repositories will now only sync thumbnails when their updates are completely synced (to stop 404s from since-deleted files)
fixed a ( 0, 0 ) resize event bug that was sometimes causing borked media containers on media viewer shutdown
syncing to a sha256 tag archive will still import the data as requested, but a popup note will explain that as everything will be imported, further syncing is pointless., and the sync will not be saved
fixed a bug in hta export if you click cancel on the hash_type choosing dialogs
misc cleanup

version 201

exported hash and tag master tables to external database files for both client and server
the client's 'mappings' table is split into 'current_mappings' and 'pending_mappings', saving about 20% of space and speed for almost all mappings operations
removed superfluous rowid column from client mappings tables, saving about another 20% of space
removed superfluous rowid column from server mappings tables, saving about 20% of space
to save time, service mapping counts now only aggregate current mappings, not current and pending
exporting to tag archive now only exports current_mappings, not pending
'all known files' queries will more cleverly discern their current/pending search domain
tag count queries will more cleverly discern their current/pending search domain
cleaned up a lot of mapping-related query code, and any ugly status leftovers from current/pending union
num_namespaces is no longer tracked for tag services
unified clientside service deletion code, fixing several orphan-generation issues
db update will remove all mappings orphans created by a recent tag service deletion
some db settings init is harmonised into one location
fixed up some external db settings init
db connection settings are now applied to all attached db files
db vacuum code is harmonised into one location
vacuum cleans up after itself better
db vacuum is now preceded by a hard drive free space check
db vacuum can now predict how long it will take
if a vacuum fails for an unexpected reason, the db will now recover better
the db recovers from failed backups better
the server db will vacuum all its databases on a backup, except any that it thinks will take total vacuum time to more than five minutes
server backup will report roughly how long it took, clientside
significantly sped up system-tag-only 'all known files' queries for tag services that have many mappings
significantly sped up system:numtags for =0 or >0 when tag services have many mappings
the status bar has some improved grammar, reporting overall common mime as appropriate
the status bar reports single-thumbnail info text when one file is selected
fixed an image cache key type-matching bug
data cache now tracks objects with dynamic memory usage
the static image rendering-cache relationship has been slightly dejanked
added a couple of checkboxes to the options->gui page to hide/show thumbnail title banner and page indicator
the manage tags censorship dialog now defaults to the 'all known tags' service on init
slideshows will not progress if the current media is either a static image that has yet to render or an unpaused animation that has yet to run through once
service content package status messages now summarise their final commit status better
this nice new line is printed to the log as a record of repo sync rows/s speed
the client will shut down faster if big jobs are running

version 200

added 'censor tag' to tag right-click menu
cleaned tag censorship a bit and moved from the gui down to the db layer
fixed tag censorship for multiple services
sped up tag censorship a bit
improved some tag censorship logic for unusual namespace situations
tag censorship now applies to siblings and parents, even in their management dialogs!
fixed the missing fit zoom when scrolling through zooms and when flipping between 100% and fit zoom
cleaned some canvas init code, which was loading bad initial zoom values due to other recent changes
restored standard static image zoom mode for cv-supported images to high, inter_area quality
cleaned up some image/animation container init code
cleaned up some image cache image tracking code
fixed ipfs pin, which was conflicting with the new ac caches
extracted client deleted_mappings to their own db table
prepped mappings processing to work with status-separate mappings tables--it is also a lot cleaner now
because of the deleted_mappings extraction, a bunch of mappings membership tests are much simpler now
upped the db cache size to a semi-ludicrous 150MB, which nonetheless seems to improve processing speed
improved how hydrus databases manage their db file locations
improved how the client and server databases backup/restore
improved some non-local thumbnail fetching code
converted some old YAML stuff to JSON
improved big job polite pausing code
cleaned up some clientside imports, and set the client to only import its server code if it needs to boot a local service
when an import folder needs to rename a file it is about to move because of a conflict, it will now add autoincrementing numbers, starting at 0, and before the file extension
added some missing shortcut actions (pan_x and set_media_focus) to the shortcut input entry dialog
tags and siblings that match the current WRITE autocomplete tags entry will no longer have doubled parents!
improved how system:limit is applied to searches

Revision #2
Created 12 March 2021 17:07:04 by CuddleBear
Updated 12 March 2021 18:24:07 by CuddleBear