added 'all known files' autocomplete caches, so manage tags dialogs' autocompletes should now always appear quickly
the now caches will be created on startup, which may take a few minutes
deleted the old tag autocomplete cache
wrote a faster initialisation routine for all known files caches
fixed an ac_cache cleanup typo
the client's (usually gigantic) mappings table is temporarily extracted to an external db file, which should have numerous benefits and perhaps a few drawbacks for now
the update will take a few more minutes, and will need lots of disk space, to extract the mappings
the server has a very similar external mappings table, so it will also need a while to update, but future admin backup calls will be a lot quicker, due to a much smaller vacuum
greatly optimised a mass-membership calculation in tag processing
reduced some db check-spam timer that was adding up idle cpu time because of recent ac_cache additions
fixed predicate parent sorting
if swfrender fails to generate a swf thumbnail within 60 seconds, the client will now dump out (rather than hanging indefinitely)
fixed some string-selection dialog layout rules
optimised some 'all known files' search queries
subscriptions will now append new urls in oldest->newest order, roughly preserving that order in the internal url cache
the 8chan thread watcher now accepts .json API urls directly
the repository processing routine will handle server_busy errors more gracefully
cleaned and refactored most media viewer 'canvas' code, dejankifying a whole bunch of related stuff and generally improving the object hierarchy
cleaned up and unified a lot of logic and responsibility for idle checking and shutdown maintenance stop time for some big jobs
ipfs directory trees are thrown up on a new checkbox tree dialog for selection/filtering
created a new download_urls (i.e. multiple urls in one popup) popup call, which the new ipfs directory downloader uses
improved a very inefficient line of sql in the mappings fetch stage of query result building that was also used in the manage tags dialog. on my dev machine, fetching 256 files' mappings dropped from 2.5s to 9ms!
regular gallery pages and 'page of images' pages now have a 'paste input' button to mass-add newline-separated queries from the clipboard
fixed a typo in new txt_tags import folder code
fixed bad autocomplete predicate sorting
fixed some sibling autocomplete search_text matching logic
some byte-based value/range presentation will now correctly convert to bytes, rather than regular decimal numbers
improved linesep splitting code across the program
fixed get/setlabel calls across the program to get/setlabeltext, so ampersand characters are handled properly
wrote an 'all known files' autocomplete cache db and prepped a bunch of code for it--it should be easy to finish next week
added a new regex practise link to the regex dialogs
os x app release should display better on retina displays
on client boot, autocomplete caches for specific file/tag service cross-references are now initialised and populated. progress is shown on the splash window
on client boot, surplus autocomplete caches are deleted
on service add, new autocomplete caches are created
on file add/delete, autocomplete caches are updated
on mappings pend/add/rescind pend/delete, autocomplete caches are updated
the new autocomplete caches are consulted for all non-'all known tags' queries
the old autocomplete cache no longer stores counts for specific file services, and the remaining associated maintenance calls are deleted
databases now start their own mainloops
databases now wait for their mainloops to finish prepping any large caches before they return to the controller
the client database waits for autocomplete caches to finish before it finishes its own mainloop
the padding around flash and the animation bar are included more accurately in some media zoom calculations, which should eliminate some general zoom jankiness and accidental 100% flash zoom coincidences that filled up the whole canvas
fixed some clientside server boot error spam when local server or booru had no port set
account refreshes that fail due to a network error will spam less to the log
fixed .txt unicode tag parsing from import folders, which was not decoding at the correct step
administrator immediate repository syncs now sync thumbnail downloads if needed
service thumbnail sync will no longer superfluously check the presence of thumbnails whose files are local
if a tag entered into the manage tags dialog has a sibling that already exists for all files, then a new 'ignore, as the sibling already exists' choice will appear
fixed an overcounting bug in 'selection tags' when importing and adding tags at the same time
fixed a typo in repository sync status text that was overcounting total number of updates by one
fixed youtube downloader, which broke with the new library on my new dev machine
the way that tags and predicates are filtered against a tag autocomplete text entry is now much faster
bumped up the default content update chunk threshold from 100 rows to 5,000, which seems to be speeding up processing significantly, with a cost to recovery latency--see how it works for you
fixed the 8chan thread watcher for boards that host content on media.8ch.net
improved the thread watcher url check logic so it won't lag with the new fix
cleaned up the ac generation code a little
'all known tags' ac counts are now summed from all the known tag services rather than calculated directly (a <= indicator for when these cases overlap will be forthcoming). this speeds up file add/delete, service reset, a/c fetch time, and general tag processing, and reduces the size of the db
ac generation code now deals with 'is the entry text an exact match or not?' better
ac generation code will now no longer produce non-exact-match siblings on an exact match search
ac generation code will no longer save half complete search text into the db as new tags
on update, the a/c cache and its helper table 'existing tags' will be cleaned of a lot of orphans, which may take a few minutes
fixed some bad unicode path parsing when importing files in some OSes, I think!
fixed some bad read autocomplete sibling substitution
fixed a bug where autocomplete predicate lists would not update if the new list was merely a reorder (which can happen in some unusual sibling cases)
fixed the tumblr parser for the subtly new API
import folders now support loading tags from neighbouring .txt files--check the dialog to set up which tag services you would like to import to
the ipfs file downloader now queries DAG object links, determines if the given multihash is a directory or other complicated object, and if so politely dumps out (handling of directory downloads is forthcoming)
added a 'multihash prefix' option to ipfs services in 'services->manage services'--it will prefix the given text to multihashes copied to the clipboard (e.g. you could use 'http://127.0.0.1:8080/ipfs/')
fixed a bad repo sync processing bug that was incorrectly assuming packages were completely processed when they often were not
all repos will have their processing timestamps reset on update, forcing a (fast) reprocess of everything they have to cover the previously missed content
fixed 8chan OP image parsing, which was sometimes failing due to absent md5s in the json
widened the splash screen a little more so repo sync text can fit a bit better
a couple of splash status texts are shuffled around
updated to sqlite 3.11.0 for windows
if sqlite3 reports at least version 3.11.0, the db will stick to WAL for large transactions (lke vacuum and content processing), since these are fast now
'namespace:*anything*' will now only appear if the a/c input has no 'subtag' component., e.g. 'character:'
import and export folders now print simple summaries to the log if they do some work
tidied up the 'pages' menu
the various hardcoded 'delete' key events are now also triggered by a backspace on os x
added a rough 'copy known urls' to the regular thumbnail menu. this is prototype, let me know how you would like this information managed and displayed in future
in prep for the new cache layer, the autocomplete dropdowns (and hence search domains of all contexts) no longer support 'all known files' cross referenced with 'all known tags'
ipfs pins and unpins can now be queued up like file repository pending and petitioned, through the regular thumbnail right-click menu, which also reports some/all ipfs pinned selection status
this ipfs action queue is similarly summarised and commited at the normal service 'pending' menu
ipfs's 'pinned', 'to pin', and 'to unpin' statuses are displayed on thumbnails with ipfs-specific icons
you can copy the focussed file's ipfs multihash or all the selected files' ipfs multihashes from the thumbnail menu's share->copy->ipfs multihash
added a .txt tag parser to the 'path tagging' import dialog--it will parse the same sort of txt files the export dialog produces
the client's new 'requests' network code is harmonised, generally improved, and now produces hydrus-compatible exceptions
updated help re the local server and boorus now defaulting to off
db can now remember service-specific filenames (e.g. ipfs multihashes)
cleaned up some overly complicated and confused thumbnail menu code
the pending menu now specifies what it is about to do more plainly
the client's local server and local booru can be turned off from their respective management panels, and from now on, the client will initialise with them this way.
if the local server or the local booru are not running, their copy/share commands won't appear in right-click menus
the welcome dialog is now a simpler popup message
incidence sorted tag lists are now sub-sorted by a-z lexicographic
pasting many tags that have siblings to the manage tags dialog will ask you if you want to always preference the sibling, saving time
added a 'clear deleted file records' button to the local file service on the review services window
idle mode now cannot naturally engage within the first two minutes since client boot
the autocomplete search logic will not count namespace characters in the autocomplete character threshold, so typing 'character:a' will not typically trigger a (very laggy) full search
putting a '*' anywhere in an autocomplete search_text will force a full search, ignoring the a/c character threshold
moved some specific 'give gui time to catch up' pause code to the generalised pause/cancel code that a lot of stuff uses, so big jobs should generally be a bit more polite
split the daemon class into two--one for big jobs that remains polite, and another for small jobs that triggers regardless of what else is going on. this should increase responsivity for a number of scenarios
fixed some bad wal failure detection and hence no-wal file creation on some instances of db cursor reinit (usually after service modification). because of now many superfluous no-wal files, existing no-wal files will be deleted on db update
some external storage location errors are improved
some internal and external storage location init is improved.
if an error is detected in the external storage location manager, it will not attempt to rebalance again until the client is rebooted
improved some upnp error catching
cleaned up some misc shutdown thread-gui interaction error spam
did some prep work on a future rewrite of daemon jobs pipeline
split up some mixed file/data/404 'stuff was missing' exception code
added simple ipfs service gui to manage and review services windows
added simple ipfs download and import to pages menu
split confused service data object into a bunch of more flexible and readable subclasses
raw url downloads are now powered by 'requests' rather than my hacked http engine, and work a hell of a lot better. more to follow
raw url downloads now pause and cancel correctly
optimised some a/c cache clearing in mapping update processing
optimised content update object -> db yield logistics
added no-daemons command line switch to disable all daemons
added no-wal command line switch to disable WAL journalling for the db
if the db fails to read a new WAL-journalled db file it has just created, it will delete the file and attempt to recreate the db, never trying WAL again
changed the way the db writes big repo syncs to disk so that the job can be abandoned much more quickly
improved local file not found error to state the directory or exact path that was the problem
fixed some socket-level error reporting (I think timeout was part of this)
the petition approval and denial client-to-server network commit is broken up into smaller steps to reduce server lag and the overall likelihood of timeouts
subscriptions will no longer add files to their 'successful files' button when the file's url seemed new but it actually was previously deleted
subscriptions will now identify themselves with an additional popup message when they encounter critical errors during sync
editing a clientside service will force a reset of the appropriate session cache, so if you change access key, the account changes will be reflected immediately
the import files dialog will no longer spam error popups on uninteresting or empty files--its 'parsing complete' statement is more verbose instead
the 1.5s gap between parsing complete and the buttons being enabled is also removed. this was supposed to be helpful, to give you a moment to check the files were as expected, but in the end it was just annoying
fixed a typo in some thumbnail error catching and regeneration
improved my network code syncing, so linux and os x releases are cleared out of many old files
split the big analyze db calls into individual table/index calls and moved them from update code to the normal maintenance routines
on vacuum, both the client and server dbs will now bump their page size up to 4096 if they are running on windows (server vacuum is triggered by running a backup)
vacuum should be a slightly faster operation for both the client and server
boosted the db cache significantly--we'll see if it makes much difference for things.
the way the selection tags control updates its taglist on increases to its media cache is massively sped up. An update on a 5,000-thumbnail-strong page now typically works in 3ms instead of ~250ms. large import pages should stream new results much more quickly now
sped up some slow hash calculation code that was lagging a variety of large operations
some hash caching responsibility is moved about to make it available for the add_media_results comparison, which now typically works in sub-millisecond time (was about 16ms before)
some sorted media list index recalculation now works faster
some internal media object hashing is now cached, so sorted list index regeneration is a bit faster
some medialist file counting is now superfast
wrote a new pauser object to break big jobs up more conveniently and reduce gui choking
the repo processing db call now uses this pauser
some copy and mirror directory functions now use this pauser
backup and restore code for the client now skips re-copying files if they share the same last modified date and file size
backup code for the server now skips re-copying files if they share the same last modified date and file size
http cannotsendrequest and badstatusline errors will now provoke two reattempts before being raised
socket error 10013 (no access permission, usually due to a firewall) is caught and a nicer error produced
socket error 10054 (remote host reset connection) is caught, and the connection is reattempted twice before being raised
the old giphy API is gone, so I have removed giphy
forced shutdown due to system exit/logoff is handled better
pubsub-related shutdown exceptions are now caught and silenced
an unusual shutdown exception is now caught
fixed a copy subtag menu typo
cleaned some misc hydrus path code
tags that begin with a colon (like ':)' ) will now render correctly in the media canvas background
some misc code cleanup
dropped flvlib since ffmpeg parses flv metadata better anyway
the 'call to' thread pool now has a delayed start, which fixes a shutdown hang in linux (and perhaps the other platforms, in one way or another), when the user hits 'forget it' on the 'already running' boot dialog
removed some console error junk on abandoned client boot
if a requested full-size thumbnail is missing, the client will now attempt to regenerate it
if a requested full-size thumbnail does not render (for instance, during a request to generate a resized thumbnail), the client will now attempt to regenerate it
some thumbnail fetching and generation logic is slightly improved
you can now drag animated media around when the inital drag click begins over the animation canvas
ditched some ultimately unhelpful mouse warping when the cursor was near the edge of the media viewer when a drag began
possibly fixed a strange tags_fts4 mirroring problem that has hit some users for a reason I do not understand
import tag options panels will no longer order their tag services randomly
fixed a bug where import folders would not save changes to their explicit tags
fixed a bug where under some circumstances, the gallery importer was attempting to display previously deleted files on the screen, causing errors.
subscriptions will now pause a few seconds after a file import error, which should solve a problem with client hanging when lots and lots of subscription file errors occur one after another
if subscriptions encounter 5 file import errors, they will pause their sync and try continuing later
pixiv manga error handling is improved
bad mime error handling is improved for all importers
fixed an important bug in the import status cache where error texts were not being stored as unicode, so error texts with non-unicode-compatible characters were causing exceptions when the session was being saved to the db or the import status frame was being loaded
existing sessions' import status caches should automatically convert their 'notes' to unicode on update
removed booru db fetch spam when booru download page has no work to do
fixed a typo in regenerate thumbnails popup message
fixed some bad event handling for the gallery page's file limit control
updated some regex defaults to have better multiplatform support and handle a wider array of valid characters
some libraries got updated for windows, to varying minor effects
BeautifulSoup updated to latest version for everything
fixed a warning from newer versions of BeautifulSoup by explicitly using lxml
updated the running_from_source help file to what is currently needed
ffmpeg.exe is also 64-bit
wrote a "reducing program lag" page for the help
added program restart to file menu, doesn't work for linux yet
improved restart code (it was used in restore database, and for some users would not close the parent process down properly)
in the media viewer, page up/down, arrow keys and mouse scroll no longer unhide the mouse
in the media viewer under Windows, dragging no longer unhides the mouse
the implicit delay in the downloader-importers is reduced, so redundant files should always stream in a super fast now
export files dialog will now auto-create missing destination directories
export files dialog will now work off the gui thread (to stop gui hangups on big jobs) and will report its export status through the export button, which will be disabled until the job is finished
changed the new tab/shift+tab shortcuts because they break panel control navigation (I want to make these editable soon):
'fetch autocomplete results now' shortcut is now Ctrl+Space (Raw Control for OS X, not command)
'IME mode toggle' is now Insert
cleaned up how control is handled cross-platform--in general, 'control' is always control for windows and linux, and 'control' is command for os x. this may be incorrect.
json dump errors now report more information about the specific object and its serialisable contents that are causing the problem
clipped sibling/parent reason entry dialog message when lots of pairs are added
fixed a bit of bad logic in sibling/parent dialog->content-update workflow
fixed the thumbnail canvas not refreshing on a canvas shrink caused by expansion of the management panel that doesn't change the thumb column count
fixed emergency boot error reporting for when HydrusData won't load
right clicking on a tag/predicate in the 'selection tags' box or the 'search' list of active predicates provides new intelligent menu options to discard/require/permit/exclude the selected tags from the current search. try it out!
this system also works for namespace and wildcard and system predicates
'predicate a' and its inverse '-predicate a' are no longer considered the same!
namespace and wildcard predicates will now remove their inverts if they are added to the list of active predicates
the way inverse predicates are generated and compared is generalised and improved
fixed 'open a new search page for tag x', which might have been broken?
quick-entering a tag in the 'read' tag autocomplete entry will now always replace with a sibling if one exists
quick-entering a tag in the 'write' tag autocomplete entry will now not replace with a sibling--you will get the option of what you want to do
when this happens, the preferred sibling is labelled on the 'what do you want to do?' popup in the manage tags dialog
also when this happens, the affected file count for add/pend the preferred sibling should now be correct
fixed client support for tags that begin with ':', such as many emoticons. these should now be parsed correctly from websites and keyboard input, keep their leading colon through network and db conversions, and display correctly (for the most part!), despite a little bit of magic to make them work
created a new TimeDelta control to handle finer tuning of optional periods
improved timedelta->pretty string code to support >60s times
manage subscriptions dialog now supports day and hour period, with min period of four hours
manage export folders dialog now supports day/hour/minute period, and the parent dialog displays that time delta appropriately
manage import folders dialog now supports day/hour/minute period, and the parent dialog displays that time delta appropriately
the thread watcher now supports hour/minute/seconds check period
folded 'search' and 'download' menus into 'view' and renamed it 'pages'
'sessions' moved from 'file' to the new 'pages'
moved 'admin' menu to 'services->administrate services'
added 'forced system:limit' to options->speed and memory panel
if several system:limits exist in a search, the minimum will be used
added a checkbox to options->downloading to replace the traffic light waiting politely indicator with text
subscriptions with failed files will say 'x urls in cache, y failed' on their dialog panel
at the gui level, media that superfluously receives a 'pend tag' instruction for a tag that is already current will discard the instruction (this improves the accuracy of the pending tag count during and after the manage tag dialog)
and the same thing for 'petition tag' when the tag isn't already current
any accidentally added 'namespace:'-type tags will be deleted on update
fixed a bad merge in the manage tag siblings dialog's autocomplete dropdown lists
the thread watcher can now deal with urls with a #54951495 post anchor at the end
refactored some controller manager code
removed some useless old code
cleaned some misc code
improved some bad old orphan deletion code
deleted some old code
finished off some old media result streaming code
refactored a bunch of search stuff from clientdata to clientsearch
rewrote some subs gui/help text to be a bit clearer, and added a help link to the dialog
you can add external client_files folders in options->file storage locations, further giving them weight
a new daemon will incrementally rebalance your files (and recover orphaned subfolders!) over your different storage locations
you can also force a full rebalance from the new database->maintenance->rebalance file storage
simplified the maintenance and processing panel
the maintenance and processing panel controls now appropriately support being set to 'none'
the 'run jobs on idle?' question is now explicit on the maintenance and processing panel
deselecting 'run jobs on idle/shutdown' will now disable subordinate controls
hitting tab on the autocomplete control now triggers an immediate 'fetch results' call
added a checkbox to options->speed and memory to completely disable automatic autocomplete results fetching (i.e. if you want to manually control a/c result-fetching only with this new tab shortcut)
the new less-laggy autocomplete results fetch won't trigger if the latest query is shorter than the cached query (i.e. you won't get lag when hitting backspace a bunch of times on autocomplete)
hitting shift+tab on the autocomplete control now disables all other key event capture, letting you enter IME without your up/down/enter presses controlling the dropdown list of results
'waiting politely' time is now indicated with a small 'traffic light'-type circle control on all downloader pages
fixed some occasional 'I'll go sit in the top-left corner of the screen and not fix my position' hover window behaviour in Linux
fixed sibling predicate collapse for the 'read' autocomplete dropdown for database results
refactored how predicates are collapsed and sorted to be a bit more sensible
fixed maintenance of local booru data use even if local booru does not receive any requests
improved some service/content update error handling in repository sync
the system:rating value-entry dialog no longer lists its rating services in random order
'open externally' will no longer show for non-local thumbnails' right-click menus
fixed audio/pdf thumbnail display for local booru, although they will be full size for now
harmonised some hex-prefix folder generation code
improved some file and directory copying code
improved upload pending popup message cleanup
added help_dir to hydrusconstants
miscellaneous refactoring
misc cleanup
extracted server-specific services and server resources from the common import path to the server import path
added swf thumbnail support--it works ok for most swfs!
thumbs for existing swf files will generate on db update for both client and server
the server will also generate thumbnails for video on update, which it seems it was not doing before
rewrote some hash comparisons in thumbnail downloading and thumbnail counting code to be a lot faster and memory efficient
fixed thumbnail count invalidation after service sync
in certain cases, fetching autocomplete tag results from the db will be slightly less laggy
if the autocomplete dropdown has cached or otherwise quickly fetchable results, it will refilter for current input as you type, ignoring the normal character delay (i.e. typing through autocomplete is far less laggy now)
fixed a couple of autocomplete tag-parsing bugs for tags with more than one colon character
fixed some key event selection bugs in the autocomplete taglist when there are no results in the list
if there are no results in the autocomplete taglist, the typically caret-moving key events up/down/home/end will move the caret (rather than being uselessly passed on to the empty taglist)
all profiling now goes through a single location
profiling now also prints extensive information on explicit code callers
fixed db profile mode printing display for the new logger
added new pubsub_profile_mode (AKA log-killer)
all menu popup display and explicit memory cleanup is done through a single location
hover windows will now not show whenever a menu is open
hover windows will now not hide whenever a menu is open (useful when trying to go 'copy tag' off the hover taglist's menu!)
a keyerror bug when force_idle_mode was hit right at program start is fixed
unexpected serverside client disconnections are now caught and further request processing is cancelled
middle clicking on an 'empty' system predicate in the a/c dropdown now will throw up the dialog to enter a value and then pass that filled in system pred onto the new search page
duplicate parents (which can occur with overlapping grandparents) are now collapsed
the upload pending popup message cancels properly after 'server busy' events
whenever the client's services are iterated at the view-level, it will now be in random order, to stop sync bottlenecks choking other services (and anything else like this that might have been occuring)
fixed a bug with double-clicking a tag selection on the the tag censorship taglist
fixed the too-fast frame timings on some videos opencv had a problem parsing
loading many video types will now be just a little faster
remote files will no longer present the right-click menu option to share->copy->file
hitting a shortcut for manage ratings from the media viewer canvas will no longer work if you have no ratings services
the base scripts (client.pyw, server.py and test.py) now have a shebang line to facilitate non-Windows use
laid groundwork for external client_files locations
all printing to the log should now be unicode safe
some other, miscellaneous file-write locations should now be unicode safe
cleaned up some now-redundant unicode-bytestring conversion error handling
the animation scanbar will now draw a little darker while the animation is paused
the client now uses the same new log object as the server, so all logged data will be copied to the terminal, and all logged statements will be timestamped
the client and server's boot and exit statements are more harmonised
server backup is more log-verbose
server backup now makes a simple copy to 'server_backup' folder--no more _backup nonsense
all serverside requests will now print a line in the form 'PORT METHOD PATH HTTP_CODE in TIME TOOK' to the log
page up and down now work (again?) for the thumbnail view. adding shift also works for selecting a page at a time
improved some animation painting logic
improved some rating control painting logic
improved splash screen painting logic
improved all other, misc painting logic--many things should flicker less or render just a little quicker
ratings hover window will no longer re-layout (causing flicker) on ratings set
improved some media canvas painting logic to de-jaggify some zooming
in the manage options dialog, tag-related gui options are moved from the gui panel to the new tags panel
added 'apply all parents to all services' option to the tags panel
the delete key now removes active search predicates and tags in the manage tags dialog
fixed webm link parsing for rule34hentai.net
if you attempt to petition multiple tags, you will now be presented with a dialog asking you if you want to use the same reason for all the petitions
edit import folder dialog now has scrollbars and will resize itself based on your monitor size
if the static ffmpeg executable is absent from the bin folder, the client will now attempt to just call 'ffmpeg' in the normal system path
fixed some 'None media' data calculation bugs when media viewer closed during very fast slideshow
fixed a 'None media' mouse position bug when media viewer closed during very fast slideshow
all wx timers will explicitly stop on exceptions (which should reduce some types of error spam)
refactored client hydrus network session manager to a better position
wrote a better wma/wmv determining test using ffmpeg
refactored cv2 (OpenCV) out of the server's import tree
fixed some server boot crash error handling
cleaned some autocomplete matches-compiling code
deletepath and recyclepath will no longer throw an error if the path does not exist
server_messages folder is no longer referenced in the code--if you have it, feel free to delete it
memory cleans up a little faster after gui page deletion
fixed a potential bug in the server's db, very important you update this week if running on Windows
improved some thumbnail page calculation logic
improved some thumbnail page drawing logic
fixed broken vertical resize thumbnail grid issue
fixed some broken 'thumbnails have changed dimensions' event layout recalculation
fixed an idiotic typo bug that was making cached thumbnail page bmps taller or shorter than needed
thumbnail fading and other general thumbnail redraws should be a decent bit faster now
thumbnails have handed off all bmp storage responsibility to the thumbnail cache, which should mean greatly reduced memory use when browsing very large results
thumbnails will now only quick-draw if they have been seen before on the same page, which should reduced some page-refresh waterfall jankiness
thumbnail media panel double buffering seems to be working better
improved some general thumbnail drawing code
the media viewer background taglist will now show petitioned tags
the media viewer hover taglist should now look exactly like the background taglist (sorted lexicographically, no more counts, and now shows petitioned)
the 'selection tags' taglist will now show petitioned tags for 'all known tags' queries
the 'selection tags' taglist and the media viewer background and hover taglists will now count collapsed sibling tags correctly
all taglists now support ctrl+a to select all
improved some generic taglist value-setting code
parents should now only expand for their specific tag services
siblings, parents, and the pending count should now be correctly recalculated upon repository update processing
added 'copy sources' to the import status window's right-click menu so you can copy bad urls or whatever to the clipboard
added 'skip' to the import status window's right-click menu so you can skip urls you don't want
added default booru for rule34hentai.net
under certain circumstances, services with duplicate names could be created--any services like this will be renamed on update, and the loophole where this could happen is now closed
improved a locale number formatting call to ask for unicode
improved newgrounds parsing--mp4/wmv files should now work
the server logger has been formalised into a proper class
fixed some bad static image initialisation in the canvas code that was causing error spam
fixed default import tag options for hentai foundry and pixiv
copy->(bmp|path|local url) will now only appear as an option if the currently focussed media is local
fixed some gallery identifier print code
frames and dialogs will now clip their initial size to the current monitor's resolution--better scrollbar support for cramped windows will follow
updated ffmpeg static builds
updated sqlite command line exes
put off sqlite windows dll update because of critical server db bug
all tag listboxes manipulate their data as sets of tags rather than individual tags
all tag listboxes report to their callbacks as sets of tags
many dialogs and other windows that use tag listboxes now deal with tags as sets
double clicking the media viewer's tag hover window now launches the manage tags dialog
the media viewer's tag hover window now includes (+1) counts like the manage tags dialog--harmonising the underlying canvas tag list will follow
lots of tag/sibling/parent code has been iterated over
ditched some misc redundant code and needlessly tightly coupled object relationships
some bizarro tag sibling dialog code has been rewritten
the autocomplete dropdown for 'writing' will now not expand search results to include parents when the receiving control isn't interested in them or it is otherwise not appropriate
when the autocomplete dropdown does have parents, they will be selected in a group with their child
the autocomplete dropdown for writing now broadcasts 'I'm done with tagging, close the dialog pls' in a better way
'activating' the sibling or parent dialogs' listcontrols (usually by hitting enter) now processes all rows that are selected, not just the first
manage siblings and parents dialogs can now take multiple initialising tags from the taglist right-click menu
removed some redundant listbox code
cleaned up and improved some tag list event processing
cleaned up some taglist right-click menu code
added system-wide mouse idle test to idle calculation. you can set this in files->options->maintenance and processing, and it defaults to ten minutes
import folders now support tag import options' explicit tags for multiple tags to any tag service
existing import folders will be updated to the new version, and if a local tag exists, it will be intserted into the new import tag options
fixed a hentai foundry page parsing bug
the deviant art downloader can now download >1024 pixel width versions of images via the download button (and a bit of cookie magic)
gallery page queries that 404 (like for a non-existent username) will now report 'Gallery 404' rather than spamming the gallery's whole custom 404 html page to the status box
fixed a bad layout flag that meant some namespace checkboxes in tag import options could remain hidden during first panel expand until first mouseover
the import status frame now initialises its status text properly
fixed last week's num_tags optimisations, which accidentally broke num_tags < x for x > 1
the client should work on both OpenCV 2.4.x and 3.0.0 (thankfully, the only difference for our purposes turned out to be some static variable renaming)
the Windows and OS X releases now come with OpenCV 3.0.0
increased the max period of import and export folders to 30 days
the launchfile/directory thread is now a daemon, so the (some flavours of Linux) client can shutdown even if an externally launched file/dir remains open
import tag options now supports 'explicit tags', which will be added to all files imported
the default import tag options system supports explicit tags, and will propagate them to new import tag options objects
the gelbooru 503 Forbidden problem is fixed
all gallery downloaders now send the page_url as http Referer header when fetching their image_urls
pixiv parsing now grabs the correct tags for the image! previously, it was fetching the user's most common tags
for now pixiv manga (which is not yet supported), will be skipped over in the gallery parsing step
fixed a thumbnail drag-and-drop calculation that meant the DnD event would not start if the initial mouse movement went in an up-left direction
bad animated gif frame timing detection is improved
the fallback animated gif frame timing is scaled back to a more reasonable 12fps
fixed a critical bug in the manage ratings dialog that meant any NULL or MIXED numerical rating controls were mistakenly creating 'set rating to 0.0' content updates
fixed an obselete call with the new Pillow library that broke video thumbnail generation
added an explicit file import test to my test suite (previously, this was a manual test, and I did not notice the failed video files last week)
fixed a critical os x crash after navigating media with the manage tags dialog launched from the media viewer
improved speed and memory use of num_tags queries a little bit
reordered the predicate filters in the main query code so that num_tags happens right at the end, where it will run fastest
removed an unnecessary filter from the main query code
removed an accidental doubling of the simple predicate filters from the main query code where the file service domain is specified
improved an important db index and made sure the db uses it everywhere it can. it should speed up most non-'all known tags' queries
harmonised how all the new serialisable JSON objects are sent to and read from the db to a handful of simple general functions
externally opening a file is offloaded to a temporary thread to avoid gui lockups on OS flavours that have delayed 'open this file pls' calls
increased the commonly used read/write 64k block size to 256k, let's see if it reduces some file fragging
clientside hydrus bandwidth reporting is a bit faster and simpler
general code cleanup
general refactoring
a long-term refactoring job that aimed to better decouple the client, server, and hydrus parts is now completely done
delineated the layout of the gallery and page of images management panels a little
updated several links in the help and the program to https
the petition content checkbox list is now significantly taller
the old YAML "default advanced tag options" is converted entirely to the new JSON "default import tag options"
created a new JSON object to store new options objects
updated the gui to handle new default import tag options objects
existing default advanced tag options data will be converted to the new system on update
the petition management panel now presents similar petitions from the same user together, with the relevant data rows in a checkbox list, so janitors can approve/deny many similar petitions all at once
the petetion management page no longer auto-loads file results for a petition--you can now see specific results for each sub-petition by double-clicking on them
the server-to-client petition object is converted from yaml to json
the new petition object now stores all the different 'content' rows for the same service/account/reason/action
small content rows are now stored and transmitted in their own object
the network now supports POST JSON requests
account modifications now work through JSON
the account identifier object is converted from yaml to json
in the path tagging dialog, regexes that do not compile will now say so, reporting their short regex error statement
fixed a bug that could sometimes occur when cancelling an ongoing gallery query
cancelling an ongoing gallery query is now reflected in the gui much faster
repositories will not burn CPU cleaning their updates if there are still pending file/mapping petitions to process
repositories will clean updates in smaller bursts to reduce the contiguous server_busy time
fixed drag and drop thumbnail ordering, which actually wasn't working but was accidentally randomly sorted when I tested it!
a certain kind of error during vacuum that can mean two different things is now caught, and automatic vacuums then suspended while the user figures out what is going on
revised some v176 update code that wasn't parsing some downloader pages correctly--now, if a downloader page cannot be updated, it will be discarded
updated to Pillow 3.0.0 on all platforms and fixed some obselete image calls
the gallery downloader is updated to the new system
the gallery downloader code is generally a bit better all around
the gallery downloader will now remember its urls and file and tag options and paused status and so on throughout a reboot
the gallery downloader supports 'get tags even if file already in db' option
the gallery downloader management panel is updated for the new object. like the other panel updates, it is a little more minimalist
the gallery downloader management panel now has the import status button so you can review your parsed urls and retry them, or copy errors, or whatever
existing gallery downloader pages in gui sessions will semi-convert on the db update
subscriptions are updated to the new system
the subscription code is generally a bit better all around
subscriptions will no longer dump out on a failed file!
subscriptions can now recognise when files are uploaded during a page-walk (which shuffles everything up by one, of course), and recover without assuming it has synced
subscriptions can now be set an initial file limit and a periodic file limit separately
the manage subscriptions dialog is updated for the new object
the manage subscriptions dialog now has the import status button so you can review your parsed urls and retry them, or copy errors, or whatever
existing subscriptions will convert on the db update
subscription popup now shows per-file download progress
boorus that use number of thumbs to advance their gallery page indices can now resume
the thread watcher now obeys 'exclude deleted' status more reliably
the 'page of images' downloader now obeys 'exclude deleted' status more reliably
subscriptions now attempt to recover their sync status when a new file is uploaded during sync
fixed a bug where the thread watcher was not applying the filename namespace to new files, only redundant ones
the old advanced_import_options dictionaries are all updated to the new import_file_options object
improved subscription-dialog-was-opened-while-subs-were-running detection and reaction
the way galleries are identified behind the scenes is now much simpler and more flexible
for now, hentai foundry downloads and subs will no longer support the specific hf filter--they will be unfiltered and sorted newest first, so they work like all the other galleries. I may reintroduce 'sort by popularity' if there is demand for it
a bit of importer status update refactoring and cleanup
fixed some db vacuum maintenance timing that was interfering with gui reporting, so the 'currently vacuuming' popup never appeared until the vacuum was complete
fixed a possible bug for import folders that have a local tag set
splash screen is no longer stay_on_top--if there is demand, I will add an option
fixed a bug in the page deleted detection code
added page hidden detection code, replaced some ambiguous pause code with it
improved pause/play code throughout the program
corrected some bad x/y status texts
for simplicity, the import_file db command now always returns ( status, hash )
for simplicity, the import_file db command no longer takes service_keys_to_tags
server will now log everything to console as well as server.log
server log entries will always be prefixed by the time they were made
server shutdown will close services more neatly
server shutdown cannot be triggered multiple times with multiple keyboard interrupts
server initialisation and shutdown is more verbose
server now accepts POST '/shutdown' requests from localhost
fixed a server shutdown bug
mousewheel events that hit the command and ratings hover windows will be propagated to the media viewer (e.g. if you click a rating, you can now scroll immediately to move to the next file without having to refocus the media viewer window)
fixed a bug where zooming out would go immediately to 1% if the media's exact fit zoom was already in the list of default zooms
added 'check db integrity' to database maintenance menu
multiple drag-and-dropped thumbnails will now be listed inside the drag-and-drop object in their hydrus thumbnail order (was previously random)
animations will now pause/play correctly during inbox filtering
most client errors now cause a 1 second wait on behalf of the caller, which should bottleneck error spam when it happens
fixed some buggy display and rescind behaviour with pre-dialog-existing petitions in the manage parents and siblings dialogs
the manage parents and siblings dialogs will now not attempt to remove/petition/rescind a pair if the entered list of pairs has entirely new entries
the manage parents and siblings dialogs will now attempt to maintain their sort order after you change some pairs
generally cleaned some parent and sibling dialog code
added option to options->files and trash for whether to use your OS's recycle bin
system:mime text now specifies specific mime group if appropriate, or otherwise lists every mime if >1 selected ('system:mime is specified' is dropped)
hydrus repositories will sometimes catch and recover from zero-length update files
hydrus repositories will catch processing parsing errors and pause when they happen
the new hdd, page of images, and thread watcher import daemons will tidy themselves up if their accompanying pages are completely deleted
import folders now have names, so you can have several pointing to the same location
import folders can filter by mime
import folders now have separate actions for successful, redundant, deleted, and failed files
these actions can be 'leave alone', 'delete' or 'move to location'
import folders now support import file options
import folders can now be individually paused
import folders' popups are individually optional
import folder edit dialog provides a path cache button, like new download pages, to let you review the paths it is remembering
existing import folders will be roughly converted to the new system on update, but they will be in a paused state, so you can check they are what you want
created new mime selection control
mime selection control added to mime system predicate selection dialog, so you can now search by mime more specifically
updated some mime system pred code to deal with this change
a single non-dragging left-click on an animation now pause/plays!
the animation scanbar will now only resume playing the video/flash after a scanning click/drag if it was playing before the drag began.
added a 'polite wait' option to options->downloading. this is temporary until I write a better clientside bandwidth throttling system
added a 'remove/petition all tags' button to the manage tags dialog to make mistakes easier to clean up
added 'confirm send to trash' to options->gui
added 'confirm archive/inbox' to options->gui
added a simple 'export tags in .txts?' button to the export dialog. In future, I will expand this to do service-specific exports and also add it to regular export folders
fixed some invalid name error handling in the manage dialogs
fixed a bug that sometimes occured at the end of an admin immediate sync
fixed a rare shutdown-order error in video rendering
fixed an animation bar initialisation error that was causing error spam on some videos/flash
general import code cleanup
cleared up admin-side petition wording and presentation
improved the threaded workflow and error reporting of the admin-side petition page
the thread watcher now reports its number of next seconds until the next check in properly formatted time
fixed the always_on_shutdown (without asking) option, which was asking anyway
removed a debug statement in isalreadyrunning code, whoops!
improved isalreadyrunning detection. it should work breddy gud now
added gallery file limit to new 'downloading' options page, which folds in the old thread checker options as well
added option to always embed the autocomplete dropdown window (rather than having it a floating window), which is now default on for Linux and OS X
manage services now supports two kinds of service reset for repositories
the service reset buttons now only fire on dialog ok
service reset will try, as cpu allows, to update its progress in a message popup window
administrators now have a 'sync now' button on the review services window that lets you catch up immediately to the service without having to wait for the normal update time (this will burn cpu time serverside, so be careful!)
fixed a bug when searching boorus with unicode-16 characters
the client updates directory is neater
'system busy' status is now shown on the status bar
'force unbusy' added to debug menu
invalid characters in export filenames will now be replaced by underscores
fixed a bug where rating services' cached file counts were not decrementing on de-ratings
rating services' cached file counts are reset on update
hover windows will pop in over video again, but will not if the mouse is _near_ the animation bar
searching for ':' in the autocomplete dropdown will no longer search the db for every single tag jej
fixed isalreadyrunning code for the Linux frozen build
the client's isalreadyrunning splash check will respond more quickly if you choose to wait
repository processing sync will no longer update the frontend, which approx doubles its speed
repository processing sync now only occurs when the client is idle, so it won't slow your browsing down
if you catch the client while it is processing, doing pretty much any action will make it quit out nice and quick
repository processing sync now occurs at a lower db level, meaning less laggy popup ui update and overall faster processing time
repository popup disappears after a few seconds once it is done--if you leave your client on all the time, you probably won't see it again.
split 'maintenance and memory' options panel into 'maintenance and processing' and 'speed and memory'
added 'cpu busy' option to options->maintenance and processing
added 'run stuff on shutdown' option to options->maintenance and processing to set whether pending db maintenance and repo processing can happen on shutdown
added 'max minutes on shutdown' option to options->maintenance and processing to limit how much shutdown processing can be done in one sitting
repositories can now sync on shutdown--they will report through the splash window
shutdown maintenance is checked and run in a better way
moved to a two-stage shutdown procedure
daemons are better kept track of, and they talk to the controller in a better way
daemons shut down in a better way
improved a bunch of idle/busy/just woke from sleep logic
improved how the controller maintains the db and memory
improved force_idle mode in the client
idle calculation is more accurate, and resets on more user-driven events
improved responsivity for processing popup
fixed a serious bug that was counting some unprocessed content as done when the service processing window was cancelled
misc code improvements
misc maintenance code improvements
fixed a bug with parentless center-positioned dialogs not sizing themselves properly
removed delete_orphans timespan param option
fixed bugs selecting and media-viewering collected media
fixed a missing thumbnail selection bug
fixed some os x dialog interaction and z-order problems when the splash screen is open
os x splash screen is no longer stay_on_top
fixed the 110.00% display bug
removed wx from pubsub code
pubsubs now route through the controller
reduced some pubsub overhead, we'll see if it smooths out some of the gritty stuff
misc thread worker pool is now managed by and routed through controller
some other thread-wx interaction is now routed through clientcontroller
db now interacts with controller directly, not through wx
all other indirect references to controller go through hydrusglobals, not wx
refactored everything wx away from everything hydrus/server except controller code
about window now says the client's network version
added media viewer mime options to file->options->media page
you can now set animated mimes to start in a paused state
you can now set the client to 'open externally' any media from a thumbnail activation
reworked the way the media viewer generalises and displays its media to obey these new options
improved some misc zoom code
flv video is now rendered natively by default
audio now defaults to an 'open externally' button
renamed the nebulous 'url' download page option to 'page of images'
moved the 'page of images' download page to the new system--it'll now remember its state through a restart, has a the detailed import status button, all that stuff
'page of images' download page now supports checkboxes to customise which types of links it will search for
you can now close and rename page tabs from a right-click menu!
deviant art parser now works for mature images
improved deviant art tag and image page parsing
added 'remove filtered files' option to file->options->files and trash
improved UPnP dialog error reporting for when external IP cannot be parsed and for when external IP is reported as 0.0.0.0
added an external ip/host override option to file->options->connection page
whenever the client or server deletes a regular file or directory, it will now send it to the recycle bin
files stored in the db are no longer read-only
existing files stored in the db will attempt to be set read&write on update
wrapped undo manager access in a lock, which may stop index bugs when a lot of stuff is going on
fixed a bug in download progress->progress gauge hook when content length was absent
removed previously entered zero-length namespaced tags (like 'wallpaper:') from client and server
moved 'client/server already running' check from db access test to more reliable process list review
improved 'client is already running' mini-dialog
added cpu percent utilisation check (if any cores >50%, dump out) to idle check
if the 'canvas zoom' value exactly matches a default zoom, it will not be inserted into the list of zooms to scroll through
the 'canvas zoom' will be displayed with hundreds of a percent accuracy
the new thread and hdd import pages will now obey page-global pause/resume events
the new thread and hdd import pages now have the 'sort by' dropdown
the new thread import page supports set_search_focus event
added import tag options info to 'getting started - tags' help page
relabelled the path tagging button on the import files dialog to something that made more sense
might have patched an unusual subscription error reporting bug
updated sqlite on windows--db should be a bit faster
updated pillow on all platforms
compiled a new python for my os x build with newer openssl library, which should fix some ssl problems people were having
added a button to launch this control, wrapped in its own frame, to the hdd import page
you can now drag and drop thumbnails out of the client for quick export!
the client will catch internal drag and drops, and won't interpret that as an import request
dragging files onto the client will now always show the 'copy' mouse icon
some thumbnail generation code is a bit quicker--resized thumbs are now always generated on their first viewing
if you use 200x200 thumbnails, the client won't bother to generate resized thumbs, saving space and time
the text entry dialog will now check its button status on a non-keyboard related text-change (e.g. selecting paste from the right-click menu)
the popup created by downloading a raw url or a youtube video will finish itself properly when the import attempt is completed, removing the pause and cancel buttons and making the popup dismissable
the splash screen will now stay on top
the password entry dialog will also stay on top
you can now copy the md5, sha1, or sha512 hashes of local files from the normal share->copy->hash menu
fixed an overly broad deleted/redundant test on md5/url pre-import status checks
options to remember size of manage tags and position of both manage tags and ratings dialogs added to file->options->gui
created new object to hold generalised import file status
moved hdd management controller to the new system
moved hdd management gui to the new system
hdd imports will now remember their import and pause status through a session change
misc import code improvements
hydrus client is getting out of the zip business--zips are no longer parsed for import nor able to be created for export
import and export code is simpler
repository file downloading daemon will now throw up a small auto-dismissing popup when it downloads files
repository file downloading daemon will respond to new downloads much quicker than previously
if a repsitory is in the process of backing up when it receives a request, it will now return a 503 'server temporarily busy' error (rather than timing out)
the client can now catch and handle these 503 errors gracefully
backing up will no longer block the client's gui, and it will no longer timeout if the operation takes more than ten minutes
some networking code is a little simpler
added 'select local files' and 'select trash' to thumbnail right click menu
'cancel download' added to file repository submenu of thumbnail right click menu
some buggy listbox resize behaviour (scrollbars not disappearing and occasional layout and drawing update fail) has been fixed
added a db table to track when a file was sent to trash
added an option for whether trashed files are removed from view
added a max age to trashed files
added a max size to the trash
added a daemon to maintain the trash
improved some generic deleted shortcut logic, so delete key will present option to delete from trash if appropriate
shift+delete will now undelete
misc trash code improvements
thumbnails deleted from the trash will accurately remove their inbox status icons
images downloaded from file repositories will accurately add inbox status icons
reduced chance of problems when trying to delete media from the trash when it is currently being rendered
further reduced this chance
removed redundant undelete code, folded it into add/delete code
the media viewer will now remove files deleted from the trash in all cases, even when launched from a file repository search
significantly improved how animations figure out when to put frames on screen. these timings should be much more accurate, slowing down only if your CPU can't keep up
8chan thread watcher now parses all files from multiple file posts
if a booru provides a link starting with 'Download PNG', which refers to the original source png, that will be preferred over the jpg (konachan and yande.re, which run Moebooru, do this)
booru parser is now a little more liberal in parsing for the image link
yande.re booru support is added by default
fixed some local/remote state code, which was breaking file repository searches in several ways
improved error handling in repository file download daemon
cleaned up manage options dialog code
reduced min size of the media caches in manage options dialog
moved thumbnail size to 'maintenance and memory'
added better error handling to repositories that cannot find an update file during processing
repositories that have such errors will pause to give the user a chance to figure out a solution
misc code improvements
fixed a bug where file repository downloads were not importing if the file had once been deleted
dropped the clever indices from last week--sqlite had too much trouble with them. I will reform my db tables next week to get around the whole issue. for now, I have reintroduced the bulky index and forced sqlite to use it where appropriate
added a test for collecting pending tag data
tags in the form 'text:', which would sometimes slip through when typing quickly, are now not processed
improved tag cleaning error reporting
improved when special wildcard and namespace predicates are provided
namespace predicates now display as 'namespace:*anything*'
fixed a bug when launching the media viewer from collected results
fixed a command hover window layout bug that was putting namespace text in incorrect places
fixed a bug that was causing the new client-wide processing phase option not to work because, ironically, its calculation was out of phase
review services will now state how many updates have been processed
review services will specify if an update is imminently due, rather than saying the repo is fully synched
fixed a review services layout bug that was misaligning text after an account or data usage update
fixed a bug in system:similar_to serialisation, so pages with this predicate can now be saved to a session
fixed the same bug for system:hash
vacuum should be a bit politer about waiting for its popup message to appear
database maintenance won't run while a repository is synchronising
reconfigured some important mapping indices in the client db to reduce search time for many common tag operations
the new indices have also sped up tag processing significantly
added an automatic db optimiser analyzer run whenever the db updates
pixiv now downloads the largest version of an image again
pixiv tag parsing improved
added support for some unusual mp4 types
the noquery media panel is removed. pages with no query status will now show a normal thumbnail panel with 0 files, rather than 'None' files.
check file integrity will now report number of missing files and incorrect files separately
check file integrity can now take a folder to move incorrect files, rather than deleting them
when you try to pend a tag with a sibling in the manage tags dialog, you will be prompted with the chance to pend the sibling instead
the rating system predicate dialog now allows for easier (and multiple!) selection of like rating parameters
the rating system predicate dialog now allows for easier (and multiple!) selection of numerical rating parameters
the rating system predicate dialog now offers correct predicates for 'zero allowed' numerical rating services
some bmp icon code cleanup
changed pause and cancel buttons to bitmap buttons
cleaned up a lot of pause and cancel code
now, when popups are paused, they will hide their details
popups that are cancelled by a dialog change (for instance when subscriptions are changed while subs are processing) will now dismiss themselves after a few seconds
during repository sync, http failures are recovered from more gracefully, and any pending processing will continue
the repository update downloading and processing loops will wait on the db and gui more efficiently
fixed a bug in the way the manage export folders dialog was testing existence of subdirectories
moved to rendering gifs with PIL by default on linux and os x, which has bad palette support but renders more reliably
the animation scanbar position caret is now visible on linux and os x
fixed some timing/resume issues the animation scanbar on os x
the old mediactrl video embed is removed
mp4, wmv, mkv are added to native rendering control, no audio yet (just like webm)
flv and flash will now have a little vertical padding on max zoom to make vertical hover windows easier to access
changed database to a faster synchronisation mode
tuned update processing daemon for more accurate time calculations and faster baseline speed, even if that knocks gui latency a bit
fixed rows/s being thrown wildly off by long pauses
made update processing daemon more reactive
improved some of the flow and generally cleaned the update processing daemon
added processing phase option to regular delay to processing of updates (this is useful if you run multiple clients and don't want them to process at the same time)
fixed e621 tag parsing
completely rewrote the focus engine behind the autocomplete dropdown
fixed a bunch of autocomplete dropdown's generally buggy behaviour for linux and os x
fixed autocomplete dropdown's display, sizing, and positioning for linux and os x
fixed autocomplete dropdown selecting system predicates with the keyboard
fixed autocomplete dropdown show status in linux with multiple pages open
improved all subprocess calls, removing interim shell step and parameterising passed arguments
improved how files and directories are launched in windows
upnp is fixed for linux
fixed hover window archive/inbox/delete icon buttons, which were spamming their commands to all open media viewers, not just their own
added a simple raised border to hover windows to better delineate them from the canvas background
updated the hover window size and position code to be a bit more reliable (still seems to bug out a bit on linux)
tags hover window should now expand sideways more reliably
it is now not possible to create nested export folders
searching for numtags < x will now include files with no tags
fixed manage boorus dialog OK in linux
fixed booru selection mini-dialog in linux
generally improved restoring database code
fixed restoring a database when you have tag archives in existing database
fixed a unicode encoding error when converting certain jobs to text (this was throwing errors in deviant art downloads/subscriptions--we'll see what was actually going on now, whether this is an error not being formatted right or something else)
started some great rewrite of management panel gui code
rearranged the download panel gui hierarchy slightly
replaced 'export tags' thumbnail menu entry with a tag archives system
replaced 'import metadata' file menu entry with a tag archives system
disabled thread dumper and manage imageboards and manage 4chan pass dialogs for now, because dumping code is out of date and completely screwed up
client will now not start in idle mode
help debug menu has new 'force idle mode' entry
idle mode is displayed on status bar
simplified client main gui status bar display and code
the flash and flv files' embed button now has a little border to delineate it from the canvas background
fixed some clientside bandwidth tracking code
removed some old networking code
made some custom objects draw themselves more efficiently
for now, manage tag parents and siblings dialogs will not show deleted rows. I will eventually add a 'show deleted' checkbox, like the manage tags dialog has
some static image rendering is slightly less laggy, particularly when browsing large images
fixed initial height of manage tags dialog launched from preview window
changed the 'search area' vs 'preview' sash gravity so the preview area won't expand or shrink on resize--see if you like it
autocomplete dropdown should now hide itself when focus goes to other programs
autocomplete dropdown should now hide itself when focus goes to other hydrus frames in os x
fleshed out new URLCache object to handle better gallery download url management
refactored some file status tracking stuff to a better system
refactored gallery page fetching to a better system
refactored gallery url handling to a better system
some redundant import checking should be much faster
improved error handling when booru image page parsing fails to find image url
refactored how tags are fetched for DA, tumblr, and giphy as part of the above overhaul
subscriptions now have a 'get tags even if file already in db' checkbox, defaulting to false, which will significantly speed up subs with redundant files
subscriptions will now more accurately obey initial limit
gallery downloaders will now more accurately obey file limit
merged some hentai foundry code into the general downloader
cleaned and refactored a lot of import page initialisation code
cleaned and refactored of lot of import management panel code
cleaned and refactored of lot of import code
created numerical rating control
added numerical rating control to rating dialog
added numerical rating control to rating hover window
updated like and numerical rating service info for future support of custom colours and shapes
fixed listbooks page deletion display
fixed manage boorus saving changes to existing panels
fixed manage imageboards saving changes to existing panels
fixed manage imageboards saving remove imageboard
improved and refactored listbook code, further separating active and proto pages
improved some listbook name conflict error handling
misc cleaning
export folders and the export dialog will also export file attributes like access and modification time
export folders will now overwrite destination files if they are a different size to the exportee
export folders can now be set to 'synchronise', and so will delete any other files in the directory. existing export folders will behave exactly as before
improved misc export and import folder code
moved folder type tooltip to an explicit bit of text on export and import folder edit dialogs
thumbnails can now display multiple hydrus file repository icons (current, pending, and petitioned) if multiple states apply
hydrus file repository icons are gone from media viewer--now there is a text list
hydrus file repository text list is added to hover window as well
exclude_deleted option will no longer hide files from file repository search results--it was more confusing than helpful
improved some error-prone logic in how advanced_import_options were being stored
ratings hover window icon background colour is fixed
imported bmps should now be converted to pngs with 1:1 colour (before, they were being collapsed to a 256-colour palette, which was showing ugly dither on complicated images)
fixed a sometimes segfault crash on search refresh on OS X
the preview window now supports a simple right-click menu
added 'open installation folder' to file menu
improved some media viewer precaching code, which should speed up some scrolling, particularly on the first scroll
fixed a frame seek error when looping long and/or large gifs with unusual palettes
improved recovery when fps reads as 0 on videos
added error detection and graceful recovery and reporting for missing ActiveX flash control
removed a lot of old imported messaging code that was slowing down boot
removed some other old library imports that are no longer needed
gracefully silenced 'application shutdown' daemon errors (they were previously spamming to console)
fixed a hover window display check that was sometimes spamming linux with edge-case errors
slideshows will now pause while the right-click menu is open, stopping bizarre full program crashes occuring on many menu items while slideshow was in progress
fixed a thread_id-sqlite-cache_initialisation problem that was breaking tag archive sync
harmonised copy/paste tag protocol in manage tags dialog with 'copy all tags' of the tags list
removed some old messaging code in client db creation
managed to completely break my linux and os x dev environments trying to update python--their release will come soon, once I've cleaned them up
removed some old cv code
updated to opencv 2.4.11 on windows
fixed a bug in trying to upload a small number of tag petitions
hover command buttons are now shrunk to exact fit size
added zoom buttons to command button hover window
added navigation buttons to command button hover window
added fullscreen switch button to command button hover window
added archive/inbox button to command button hover window
added delete button to command button hover window
added generic close window button to command button hover window
added tooltips to all the command buttons
cleaned some misc canvas code
hover windows now fit better, without overlap
moved media info strings to top of media canvas, so they hide behind hover header
hover windows will now not cover webms or gifs when the mouse is over the media container
hover command buttons now work for inbox filter, including back/skip buttons
inbox/archive button feels and works correctly for the inbox filter
updated manage options dialog's 'default file system predicates' mess to use the new cleaner predicate panels
all file system predicate defaults have been reset to default
added system:num_pixels to file system predicates default panel
fixed the system:num_pixels predicate being broken on options save
removed the ratings system predicates from file system predicates default panel
fixed misc system predicate bugs
moved yaml->json conversion forward:
settled on object code
added built-in compression to serialisation
added db tables and access code to support it
fixed a little layout mess in the edit custom filter action dialog
created a rich 'shortcuts' class for storing shortcut->action information that can be easily expanded to handle mouse events as well
new shortcuts class works on new json storage rather than yaml
moved custom filter shortcut action storage and general handling to new shortcuts object
moved old favourite_shortcuts tests to new code
cleaned a lot of the custom filter dialog code
rewrote the custom filter setup dialog to work more like other dialogs--now all changes are saved on ok, and save/save as/delete is replaced with add/delete
auto-creation of 'previous' shortcuts is removed, as all changes are saved to shortcut sets anyway
removed the ratings filters--the numerical filter may make a return in another form, your thoughts would be appreciated
fixed a cache counting bug when archiving redundant files during import
sped up file deletion a little bit
added an inbox cache to reduce laggy inbox checking, particularly search result fetching on dbs with large inboxes
fixed subscriptions that have no initial file limit, please reset your sub url caches to fix in these cases
did a lot of server/client database merging and refactoring
improved some database error handling
fixed an error in file repository superbans
optimised some critical db code
sped up tag censorship filtering
added a db profiling mode, accessed from the help menu, that will dump copious db profiling info to the log
misc code cleaning
improved some popup mesage print sync timing
rewrote some server auto-setup code to deal with slowly-starting server
put a nice big red warning at the top of the 'running your own server' page
added system:num_pixels for megapixel searching
system predicates height, width, ratio, and the new num_pixels are now collapsed to a super-predicate, system:dimensions
system:rating will now only show if you have some ratings services
the system:rating dialog will now hide like or numerical ratings if you have no services of that kind
complete refactorisation of system predicate dialog
some misc system predicate bugs and bad panel event precedence fixed
shortened all predicate spinctrls
improved how system predicates are stored and used for a search
moved some view menu items to the download menu and the new search menu
all of the download gallery pages are added to the download menu
added some explanatory text to the pixiv dialog
the tag uploading process will more carefully prepare its subupdates so that mass allocations of the same tag to many files will be split into smaller chunks, avoiding connection timeouts in these cases
hover windows will no longer hover over flash files
tag hover frame will resize a little based on the canvas width
the twenty pixel mouse warping when the mouse starts dragging close to the edge of the media viewer will no longer occur on flash files
the canvas buffer for flash files is expanded to five pixels either side
the hover tags window will now update itself when the media's tags change
new hover window for details and commands, up top of the media viewer
moved the old popout windows' buttons to the new hover window
added hover window commands for the normal browser
cleaned up some general canvas code
index string will now display with a slash in all cases (it was a backslash for windows, for accidental reasons)
file limit for subscriptions and download pages now has minimum value 1
searching for number of tags while tag censorship has some namespaces will no longer throw an error
moved boot and exit code responsibility from the splash frame to the controller
improved the boot and exit code generally, including dragging feedback lag and error handling
improved some thread error handling
fattening service info won't trigger on client shutdown quite so much, which should speed up shutdown a little
added prototype hover frame for tags in media view
hover frame shouldn't show when a dialog is open
manage tags dialog launched from media viewer now has a delete button
subscriptions now have an 'initial limit' variable, defaulting to 500, that limits the total number of files the subscription daemon will look for on the initial sync
added a similar file limit spinctrl to gallery download pages
updated layout of import files dialog
cleaned up some ugly global variable scope stuff
fixed initialisation of advanced import options in file import dialog
made a good start to better object serialisation
subscription and repository sync daemon jobs that stop due to a dialog-driven change during processing will now cancel themselves after five seconds
fixed namespace (e.g. 'series:') tag censorship
fixed the fullscreen switch bug that was breaking an (initially fullscreen)->(non fullscreen) media viewer
fixed some search logic (some system predicates were not firing when there were no regular tags present)
removed some artificial delays on daemon db access, let's see if it chokes anything
harmonised a bunch of client and server controller code
created a common controller class and merged a lot of the client and server controller code into it
added a possible solution to the manage tags dialog next/previous buttons crash
completely overhauled temporary file management throughout client and server
removed old temp folder
copy files to clipboard will now copy the database's exact file paths (rather than copying to a temp folder beforehand), making it a lot quicker
the import folders daemon will no longer attempt to import (and hence spam errors about) zero-length files
added one-time tag archive sync button to the 'perform a service-wide operation' dialog in review services. this lets you add or delete the archive's tags by namespace
revised the tag archive sync code a little so it syncs to non-local files as well, if the hashes can be lined up
added hash cross-referencing to archive sync code before maina processing step, often saving a huge amount of time
fixed a parsing error with advanced content updates at the gui-end
added GetName and IterateHashes to hydrus tag archives
added default 'similar to' max hamming distance to file system predicate defaults in client options
fixed up the default text display of the file service system predicate
added some text to the sort/collect options panel to better explain what the big listbox is for
fixed a popup numerical typo when uploading files to a repository
fixed an utterly bizarre bug caused by the import files dialog that was screwing up subsequent dialogs' layout and size, apparently by wx voodoo
refactored all the functions and classes of hydrusconstants to better locations
misc other refactorisation
misc cleaning
renamed hydrusdownloading to clientdownloading
changed all default { } and [ ] parameters to None. I don't think it mattered, but just in case for the future
moved some ugly global variable stuff to better places, hoping to do more in future
added an 'idle' updating processing mode that churns through tags four times faster but makes gui laggy. it will kick in whenever you haven't done anything in thirty minutes
improved some gallery downloader status texts
gallery downloader will no longer wait 5 secs before fetching first page of urls
fixed a critical race condition that had a chance of causing complete and unrecoverable gui freeze in the gallery downloader code
fixed rendering of some bizarro pngs that CV couldn't understand
fixed tag parent management for admin users
fixed up some tag parent dialog code
tag sibling dialog now accepts single reasons for multiple pairs, like the tag parent dialog
improved error state on rendering video that reports resolution of 0x0
removed looping error reporting on certain static image rendering errors
fixed an error when some thumbs were 'collected' while fading in
tagboxes now select on home/end and support numpad versions of page up/down
fixed a radio button initialisation bug
boosted the initial size of the 'set default advanced tag options' dialog so the collapsible pane is more visibile
silenced a upnp error when external ip can't be parsed
reduced some tag dialog previous/next button lag
general code cleanup and refactorisation
gui flag refactorisation and unification
refactorisation of most non-constant stuff from constants files
refactorisations of media mixins class in prep for a general rewrite of that code
improved client initialisation of default options, boorus, and imageboards
No Comments