Changelog 50-99

version 99

added backup database menu option
added restore database menu option
gone back to wx 2.9.5 for now
fullscreen scrolling should be a bit smoother now
fixed sessions for 'open selection in new window'
new regex shortcut for filename in the 'add tags before importing' dialog
fixed dialogs cancel-closing on an unfocused 'enter'
finally fixed the animation scanbar vertical sizing on preview screen
animation scanbar now shows gif frame rendering progress more smoothly
test.exe now included in windows release
some frame parents code cleanup
a tiny bit of gif code cleanup
deleted some old server code
cleaned up import pause/cancel code
improved hydrus's thread-communication objects
cleaned up downloader code and logic and gui display

version 98

update to wxpython 3.0
you can now add tags when importing from a zip
you can now download arbitrary urls from download->a raw url
'select files to import' dialog has massively improved file parsing, with:

new gauge location
queueable parse jobs
paths that stream as they are parsed
pause/cancel buttons

new 'check credentials' button in manage services dialog
new access_key_verification server request
fixed a very important bug that affected normal tags when adding parents or siblings
fixed a bug where old pending parents would not be remembered when new same-child parents were later pended
fixed a similar bug where rescinding parent pends would screw things up
pending counts will new correctly update on a sibling or parent pend rescind
parent tags no longer (incorrectly) remove when:

removing 'just this file' child tags in 'add tags before importing' dialog
removing child tags in custom filter

some youtube videos were not downloading due to a redirect issue. it is improved, possibly fixed completely
collapsible panes, like 'advanced import options', will refit dialogs on collapse/expand
fixed an bug in the way certain server errors were handled
fixed a serverside bug for GET account when using account_id as identifier
fixed 'modify petitioner' button in petition pages
fixed a bug that was preventing clearing of temp path on client startup
cleaned up some credentials code
some general code grammar improvement
reworded some help
reworked account identifiers
os x

now has an app icon, woop
os x page closing logic improved
collections checkbox dropdown seems to work, now
fixed segfault on popup message close
thumbs loading from initial session seems to draw properly, sometimes
fullscreen no longer causes segfault whenever you zoom or scroll

version 97

initial os x test build
collect typo fixed
'remove all mappings with specific namespace' added to advanced content update dialog

version 96

went over _all_ my help (about 200KB of text!), cutting down waffly paragraphs, rewritting bits that were stiff or unclear, updating screenshots, and adding several new sections people suggested
did a complete rewrite of how the client stores services in its database, streamlining everything
several bugs exposed by the services rewrite fixed
several moments of weirdness exposed by the services rewrite made more sane
new version of sqlite (3.8.2)
new version of python (2.7.6)
cleaned up a debug print statement that was spamming gauge numbers to the log
when loaded from a session, hdd import pages will now hide their processing panel
fixed namespace sorting to manage numbers again
collapsed the many different download gallery page classes to a single one
started an important rewrite of how downloaders are going to work in future
improved testing framework's pubsub testing support
subs sync daemon now restarts correctly after a dialog-driven pause
parents now obey siblings!
siblings and parents are updated on forget pending
got AMP message between persistent and temporary peers working

version 95

pages launched from a session will no longer run their query on startup; they will remember their thumbnails
import pages added to gui sessions, although they do not yet remember their progress
the way session pages are started is improved
advanced content update dialog simplified and rearranged
advanced content update db code written for copy operation
advanced content update db code written for delete operation (local service only)
advanced content update db code written for delete deleted operation ( local service only)
made first version of IMMessage objects
added network version checking to hydrus's AMP
better AMP error catching and reporting added
AMP tests moved further forward
tightened up AMP network code, fixed last bugs
cleaned up a huge pile of 'expiry/expires/expiration' semantic mess
same for some mapping_ids/mappings_ids stuff
removed predicate knowledge from media display classes, which was just a mess, really
corrected a weird param order in sessions
general code cleanup
several old instances of tuple-list yaml compromise removed
fixed a session bug in the testing code because of last week's manager management change
fixed a hydrus session bug for accounts with no expiry
fixed a very important server-side network version checking bug
added client-side network version checking

version 94

reorganised the 'add tags from path' dialogs so the regex buttons made a little more sense
when deleting a file, the current focus will pre-defocus
'confirm close client' dialog will now auto-yes after 15 seconds
fixed a bug when POST hydrus queries couldn't return any data
'failed to update repository' message made simpler
added tuples to pyyaml's safe_loader (which was more of a pain than I expected)
manager management improved, with changes reflected across all the code
session management prototype is done
query pages can be added to sessions
default startup session can be set in options
sessions are managed with new submenu under file
creating registration keys will show them in a new, more helpful frame
same for initialising a server
and for auto server setup
auto server setup has cleaner error reporting
new service admin accounts now have different access keys to the server admin account
fixed up testing framework to test AMP a bit better
IM session manager written
server db support for new session manager added

version 93

can now undo/redo all tag operations except rescind petition
can now undo/redo file pending, rescind pending and petition
improved undo/redo description generation
fixed a bug that was showing extraneous undo/redo actions that did nothing
review services frame's permissions strings will size a little better
new 'perform a service-wide update' button on review services frame
started tag-specific dialog for this new button, and began db code that'll execute its stuff
a little code reorganisation
fixed a bug in the export dialog that was throwing errors when trying to overwrite read-only files
moved a number of old dialog error messages to the new verbose popup system
when accounts are made stale (as part of client version update), they will no longer lose their privileges
fixed the log page to deal with (ignore) gauge messages
fixed a db problem in my testing framework
AMP IM support moved way forward
IM manager first draft done

version 92

encrypted instant messaging framework started
encrypted instant messaging test added
encrypted instant messaging trust framework started
started an AMP framework for encrypted instant messaging
tag A/C exact match parameter added
tag A/C exact match parameter test added
tag A/C logic reorganised and improved
tag A/C now gives exact matches for queries shorter than the A/C threshold
tag A/C threshold default set to 2
fixed an important architectural bug in pubsub
generally cleaned up pubsub code
'popup messages sometimes not updating' problem is fixed
a bit of initialisation cleanup to make startup behaviour more reliable

version 91

improved how accounts are identified in the server
rewrote how serverside session data is managed
added monthly data reset to session data
fixed session update on account modification
fixed a bug in accountidentifier generation from accounts
fixed a bug in account-multi-session tracking
server session manager unit test added
reworked testing to take arguments to only run specific tests
changed all file downloads to be more memory efficient, either by gui or subscription
made download testing fail better
added deviant art unit test
fixed deviant art thumb parsing error
fixed deviant art fullsize image parsing error
undo system reworked in prep for first prototype
undo/redo keyboard shortcuts (ctrl-z and -y) infrastructure added and db updated
undo works for fullscreen
a bunch of fullscreen keypress processing improved
undo manager testing added
added upnp option to all services
added upnp daemon to maintain upnp serverside
'copy all tags' and 'copy all tags with counts' added to tagslists
rejiggered a bit of the app startup, including db password check
found a better error noise
in fullscreen, the mouse cursor will no longer auto-hide when over an animation scanbar

version 90

client db unit tests added:

4chan_pass
autocomplete_tags
booru and boorus
downloads
favourite_custom_filter_actions
imageboard
import_folders
md5_status
media_results
namespace_blacklists
news
pixiv_account
improved services
sessions
shutdown_timestamps

fixed modify account
fixed modify account again!
made account_info respond a lot faster with large tag counts
neatened some client db code
youtube 'file already deleted' error now reported correctly
youtube general error reporting improved
fixed multiple-computer-single-account data use reporting
moved file parsing from nested dialogs to a single popup message gauge
fixed an export folder path-display issue when an install moves location
improved how some file downloaders process their files
added prototype file download status bars to all download management panels
fixed a bunch of tests that were cleaning up inelegantly

version 89

fixed blacklist manager for numtags
fixed a sql typo for numtags
delete from fullscreen menu fixed
fixed a unicode error-handling issue
sped up autocomplete tag fetch for large result sets
improved the way autocomplete dropdown scrolls
vastly improved efficiency of how all listboxes draw
the server will no longer start if something is using the server admin port
i.e. you can no longer run multiple copies of the same server by accident
improved client and server port change (service shutdown and restart)
improved testing framework to handle fake db reads better
improved testing framework to manage client and server sessions
several network request unit tests added:

access_key
account
account_info
account_types GET
account_types POST
file_repo file
file_repo thumbnail
init
ip
news
petition
registration_keys
services GET
services POST
session_key
stats
update GET
update POST

fixed clientside permissions exception
several db unit tests added:

import_file
system_predicates:

age
archive
duration
everything
file_service
hash
height
inbox
local
mime
not_local
num_tags
num_words
ratio
similar_to
size
width
limit

fixed a bug in system:age?
fixed a bug in system:duration<0
fixed a bug in system:duration?0
improved num_tags logic
fixed system:num_tags<1
fixed system:num_tags>1
fixed a hash filter bug for system:similar_to
fixed system:similar_to for unknown hashes
fixed system:size Bytes
updated linux.html and running_from_source.html in help

version 88

moved server daemon start to a better place
cleaned up more server db code
fixed getnumpetitions
fixed getpetition
reworked modifyservices and fixed server request
reworked modifyservices dialog to work with new system
server service options now managed from modifyservices dialog; old options dialogs deleted
auto-repository setup restored and fixed
harmonised edit_log actions all across the code
added namespace blacklist manager
added namespace blacklist dialog
integrated namespace blacklist into:
- search by tag
- search by tag count
- creating media results (thumbnails)
- autocomplete tags read
- autocomplete tags write
fixed a problem with deleted tags not being applied correctly on initial thumbnail display
started a server unit testing framework
added 'running in linux' info to help

version 87

misc:
- fixed system:untagged, which was doing numtags>0 by accident
basics:
- moved client local service to twisted
- moved server to twisted
- cleaned up a whole lot of unrelated server stuff I haven't touched in a while
- fixed some misc typos
details:
- built twisted server framework to plug into hydrus
- changed hydrus authorisation header
- changed sessions to manage accounts, reducing server db load
- reworked session cookie to be neater
- cleaned up a bunch of server code
- fixed how certain server errors were being printed to the log
- reworked response_context to a body/path dichotomy to reduce cpu+memory on file requests
- collapsed a bunch of GET POST requests to the same path
- reworked server_service_identifiers to be port-independant
- improved data use logging to support new session management system
- moved session management to new system
- moved error management to new system
- integrated twisted thread into hydrus controller
- reworked root and favicon
- reworked local file and thumbnail requests
- reworked restricted service requests
- reworked admin service requests
- reworked repository service requests
- removed manage options query; it'll be rolled into manage services admin queries
- updated serverdb to manage with new system, harmonising internal and external requests to one workflow
- made server-side data use tracking simpler
- added Content-Type header to most requests
- improved how key registration, init, and account GET requests are handled client-side
- harmonised how account_keys are created server-side
- moved server management from serverdb to servercontroller, with better pubsub restart
- moved several useful functions to the new serverconstants.py
- fixed a max_age sessions issue

version 86

timeout on connections improved
rewrote the objects behind the dumper
the dumper should now select media properly again
closed pages now timeout after an hour
upgraded to new version of sqlite
fixed client port bind detection and error reporting
you can set the client's local server's port in client options
local server port changes will happen when dialog changes
unified the upload and youtube download popup messages into one popup
fixed changing thumbnail size (stupid typo!)
added 'database->regenerate all thumbnails' to fix certain thumbnail errors
add a popup gauge for regen all thumbnails
fixed yesno dialogs, which were showing a neutral 'Cancel' button rather than a red 'no'
the upnp dialog's buttons now work, have fun with it!

version 85

updated to new version of wxpython
several key_down events moved to char_hook
rewrote collapsiblepane class to remove weird refit and scroll behaviour
a problem with adding service via a registration key is fixed
tag upload pending is granulated into separate queries, to reduce server and client lag
uploads moved to an asynchronous popup message with a gauge, rather than the modal progressdialog
reorganised file upload content_updates to be a bit smoother
fixed a redirection bug that wasn't obeying schema changes such as http->https
updated e621 url to https schema

version 84

switch fullscreen button added to fullscreen canvases
messaging stuff is disabled for now
gif scanbars now fill up as the frames render
pngs with transparency are no longer drawn with black background after first viewing
made a small change to listbooks to correct some gui-weirdness, particularly in local options
autocomplete dropdown windows will now hide-and-reposition on parent scroll events
autocomplete hide-and-reposition waits 250ms rather than 100ms, making it a little less flickery
a graphical bug related to hitting end on a large search is fixed
upnp framework started
services->manage local upnp started
new messagegauge added
fixed mp4 import
youtube url->formats added
youtube format chooser dialog added
youtube downloader thread added
youtube gauge popup added
youtube error handling improved
youtube num_bytes_done added
youtube unknown total_num_bytes handled
youtube cancel button added
youtube right click dismiss throws up a yes/no dialog to decide whether to cancel the download
started manage upnp dialog
closed page undo added
closed pages will be paused and quiet
pause import folders option added
fixed an invalid index drawing bug after removing certain media
thumbnail waterfall improved
tags are limited to 1024 characters

version 83

sort by longest fixed for files with no duration
reverse sort by rating fixed for files with no rating
sort by largest, newest fixed for files with unknown size or timestamp
sort by unknown/absent values sorts more accurately
search by num_tags and min_num_tags fixed
remove media thumbnail update bug fixed
currently viewable indices screwed up after a remove media bug fixed
remove all media entire black screen bug fixed
delete media redraw optimised
several message print unicode/raw byte errors fixed
scrollbar position calculation improved, so 'black rectangle on scrolled thumbnail canvas when num_cols changed after resize' bug should be fixed
'tiny black lines on small slider drag' bug fixed
moved a bunch of right_up events to right_down, see what you think
popup message cleanup is better on shutdown
pubsub typeerrors handled more gracefully
improved how options init and update works
client will now remember the restored size, restored position, maximised state, and screen the gui was last left at
client will remember all the same details for fullscreen, separately
these sizes _should_ rescue from offscreen if you disconnect/reposition a non-primary display
shift+home/end now scrolls and selects, rather than only buggily scrolling
I fixed a content_update bug re petitions
'show in new page' pages will no longer show search subbox
added hentai foundry tests
fixed hentai foundry title and creator tag parsing in certain cases
fixed some related html unicode issues
initial value on tag siblings dialog fixed
tag siblings and parent dialogs now start sorted by the right column (parent/older sibling)
setting a default collect will no longer cause a nasty crash!
setting an orphaned default collect will no longer cause problems
deselect thumbnails has improved focussed media management
can now select inbox or archive from right click menu

version 82

a bug where slow search results would sometimes appear after search predicates were removed has been fixed
a lot of autocomplete gui- and db-blocking reorganisation
searches are now entirely asynchronous to gui thread
searches are split into two granular phases, and are cancellable during processing
simplified system predicate storage
consolidated all system predicate filtering to initial db search
huge improvements to how the thumbnail canvas is sized and extended and drawn to
numerous fixes and improvements to how thumbnails are drawn to screen
reworked how collect works
and sort
the raw cpu time behind sorting in muuuuuuuch faster
clarified my listening media class
added another optimisation to canvas resizing
big improvement to canvas redrawing and refreshing
important fix in how client figured out what to draw when clicking in whitespace
thumbnail fade is much smoother
thumbnails should now generally draw a little smoother
selectall and selectnone's thumbnail fade is less cpu intensive
cleared up a number of collect related selection and index bugs
fixed a media identifier issue
all import-related append and drawing is muuuuch faster
flicker bug on small appends fixed
fixed a page up/down drawing bug on the main thumbnail canvas view
fixed an unknown-timestamp thumbnail right click issue
made my internal media data storage system a lot simpler
new sortedlist class to make some media stuff easier to manage
reworked how new media is added to a page of thumbs (usually import-append, but the new system supports collect- and sort-sympathetic insertion):
-calculating combined tags on the left can be up to ten times faster
-integrating new files into the sort is much quicker
-integrating new files into the collect is much quicker
-improved how new thumbnails are decided to be drawn
trying to archive or inbox more than one file, you'll get a yes/no dialog to confirm
started db testing framework
db tracebacks improved
slimmed down content_update data-side processing
a couple small code fixes
archive and inbox no longer remove from the search if the opposite system predicate is set
system:rating fixed
fixed a bug where you could middle-click-download files that had no known source
couple of bugs with system:ratio fixed
system:ratio string representation is 16:9, rather than 1.777779
gui thread is more intelligent in telling non-gui threads that it is busy

version 81

mp4 added
mp4 mime search added
wma added
wma mime search added
wmv added
wmv mime search added
changed to video and audio icons, rather than one for each mime
popup messages now dismiss on RIGHT_UP rather than RIGHT_DOWN
moved many more messagebox errors to the new popup error system
cleaned up clientgui's superfluous error handling
rewrote error redirection to new popup system
more error overhaul
print statements overhaul
better db updated statement, through popup system
traceback show/hide button added to new popup error box
copy button added to new popup error box
added a 'x more messages/dismiss all' popup when there are more than ten messages to display
review services refresh account button will disable until the server replies
file system predicate dialogs have better initial focus
streamlined server file import and improved respective data usage calc
made internal options storage a lot more efficient
cleaned up a little cache code
export to zip improved to new file handling system
upload to repo improved to new file handling system
some db write priority stuff switched around and renamed
streamlined a little client db workflow
neatened client db exception handling
db updated message is now a popup
removed some orphaned methods in client db
client file and thumbnail server requests made a bit faster and neater
removed db loop from export and dump file fetch
neatened some client and server file 404 exception paths
further cleaned up the way file paths are calculated and fetched
further cleaned db in relation to files, and removed the old mime cache, for both server and client
removed orphaned db export infrastructure
add_thumbnails in client made simpler
improved http response parsing
refactored a bunch of http response processing
new exception display system added for caught exceptions
fixed a shutdown hang on certain delete orphans failure

version 80

manage tags and ratings dialogs have their initial focus corrected
custom filter now obeys tag parents for tag actions
fixed an annoying as hell multiple-page shared-thumbnail visual-selection-status bug
added e621 test
fixed e621 tag parsing, including new species namespace
might have fixed a missing thumbnail error
fixed system:hash
fixed some repo file downloading stuff
fixed some general db fetching file info stuff
made x files imported from y message a little slimmer
fixed minsize for the main gui frame
added better initialsize for the fullscreen frame
putting a thread url into the thread_id part of the dumper will auto-convert to the thread_id
added dumper's multipart form data generation test
added generic multipart form data generation test
fixed dumper unicode issue
error reporting in dumper is a little better
streamlined how mime is calced
massively improved how mp3, flac and ogg are parsed and validated
massively improved the internal import files workflow to use less memory and hdd reads and writes, particularly for video
similarly improved server-side import workflow
the way temp dir works is improved
sped up server file/thumbnail fetch
reworked how thumbnails are fetched in a couple places
started rework of how options is stored
wrote mp4 properties parser, but no mp4 just yet

version 79

popup messages will now report whenever a subscription or import folder successfully imports some files, with a button to show them in a new search
popup messages now wrap
slightly better error popup
many more errors reported through error popup
old logging system switched over to new messaging system
popup message manager will only show ten messages at once, now
selectfromlistofstrings now supports enter key to select
completely reworked how dialog cancel works; absolutely all dialogs should now close with escape key
reworked how a bunch of dialogs do ok
a huge amount of dialog refactoring
made ok button initial focus of all dialogs
couple of bugs in service options dialog fixed
rejiggered some button names and focus behaviour a bit more
some classname refactoring
muchly improved string->unicode handling
improved timestamp generation
fixed a clientserviceidentifier->text bug
reworked how namespace cache in tagsmanager object is calculated
improved instantiation of noneablespinctrls
fixed a bug in the thumbnails download daemon
changed the way daemons wait for the db, much to the better
moved daemons out of the db object
rejiggered writedaemon synchrony so exceptions work
fiddled around with some help links
default, non-maximised size of client is a little more comfortable
custom filter now has a popup that'll let you change the custom actions mid-filter
server now uses new synchronous logging system
fixed an options save bug for server
updated server diagram in help
added test for dialog selectfromlistofstrings
added test for dialogyesno
made a framework for testing that requires network stuff
made a newgrounds test
fixed newgrounds swf parsing
made a framework for testing that requires file reads and writes
fixed a graceful-exception bug in mime parsing
added test for synchronous import_folders
added test for delete import_folders
wrote a test for importfolders daemon
import folders no longer delete or reattempt failed imports; they'll just ignore them
import folders are deleted on update, since old objects are obselete
import folders won't try to do zips any more; they'll just ignore them
rejiggered how import folders does its path parsing to remove mime calc cpu usage

version 78

expanded parents testing with a namespace example
made getgstvcp in tags manager only return numbers for vcp
refactored ManageDialogs to their own file
refactored Exceptions to their own file
improved my testing framework so it can do wx gui elements
added test for dialogchoosenewservicemethod, fixed a typo
added test for dialogfinishfiltering
added test for dialogfinishratingfiltering
those two finish filter dialogs are a little simpler, now
set up a system so dialogs are a bit simpler, in terms of button event processing
added test for dialogfirststart
added 'add parent to tag' on tag right click menu
added 'add sibling to tag' on tag right click menu
made manage parents/siblings dialog focus selection more intelligent
'audio - any' mime search fixed
reworked audio embed container significantly
added embed button
all noisy mimes are now protected by embed button
you can now rescind pending file uploads or petitions from thumbnail right click menu
fixed a redundant bit of code in process content updates
in modify account dialog, set account expiry to 'does not expire' fixed
logging now happens a little more synchronously for client
popup messages prototype done
popup message for normal text
popup message for some errors
fixed a small redundancy bug in data cache
fixed a localisation bug when converting numbers (1234) to formatted version (1,234, 1.234, 1 234)
improved the code behind 'open selection in new page' in two ways

version 77

tag parents manager unit tests done
completely rewrote parents pair generation
rewrote parents pair retrieval to be much faster
parents manager is much more resistant to loops
in two ways!
loop detection fixed in manage parents dialog, as well
fixed how deleted parents are applied to combined service identifier
fixed how deleted parents are applied to specific service identifier
tag manager unit tests done, and everything was working ok!
shuffled objects around a bit, unified some tag code
tag manager merge unit tests done, and everything was working ok!
tag manager merge code simplified and result rewritten to a new, simpler class
small bug in how preview canvas fetches tags fixed
finished off import folders daemon
added import folders gui
added flac support
added ogg support
added a secondary layer of mime detection for greater accuracy
added a tertiary layer of mime detection for greater mp3 accuracy!
fixed an offset bug in mime detection
simplified and unified client/server file info parsing
import error handling improved
neatened how special thumbs are stored, refreshed, and retrieved
neatened how the bmp->png conversion happens
improved hash->mime caching for both client and server, to speed up http retrieval time for the new filetypes
better mime fetch in dumper, too
improved debugging messages for a bunch of my custom objects

version 76

made unit testing framework
tag siblings object now tested properly
hydrusdownloading functions now tested properly
added simple mp3 support to client, with external launch or embed
added search by mp3 mime
added mp3 to server
higher precedence deleted tag siblings now overrule lower existing pairs
tag sibling closed loop pairs ( a, a ) are skipped
neatened predicate sibling matching in two ways
improved tag autocomplete matching logic
skip logic improved for like ratings filter
skip logic improved and button added for numerical ratings filter
did a little work on reducing memory usage
reordered data cache's memory purge, which will make huge images load better
fixed a zoom bug with pdf buttons
added some select all, select none and refresh to thumbnail right click menus
reorganised menus a bit anyway
fullscreen tags on the left are now sibling-collapsed

version 75

fullscreenpopup window added
fullscreenpopup window can be dragged about
fullscreenpopupfilterinbox
fullscreenpopupfilterlike
fullscreenpopupfilternumerical
accuracy slider moved to popup
compare same image until done added to popup
keep on left/random/right added to popup
'don't ratings filter this' added to popup
that annoying as hell thumbnail selection drawing state bug is fixed
display of flash with only one frame fixed
downloaded tags now work again, sorry for the disruption!
deleted tags will now always show in manage tags dialog
fixed the various problems that were stopping DELETED->PENDING->CURRENT working for tags
fixed the children can only have one parent bug
fixed a deleted tag parents bug
a little counting logic improved in the special deleted_pending case
fixed a typo re uploading file to a repo
main guis statusbar now has a little 'db locked' indicator

version 74

flash scanbar added
made new mediacontainer class for media canvas
extracted scanbar to separate class
made animation frame tracking and control more sensible overall
previous_frame/next_frame shortcut now works for flash as well
fixed age predicate for > hours
fixed an important 'all known tags' A/C count cache bug when multiple services have the same tags for a file
if you have resolve_petitions permission for a tag service, tag siblings and parents petitions need no reason
slight change in how un-namespaced totals are calculated in A/C read
collection now obeys siblings
thumbnail upper text now obeys siblings
fullscreen text now obeys siblings
bit of siblings code cleanup
a little changeup to how showselectioninnewpage collects its media
/asp/, /gd/, /lgbt/, /vr/ and /wsg/ added to default dumper support
trying out making the server non-daemon, to see how that manages

version 73

every file operation from fullscreen was broke due to a single-character typo that got propagated
fixed local ratings, which were broken five different ways
ditched the deleted_pending concept, which was actually making things more complicated
possible fix to review services opening on bottom level
dumper no longer says it is still dumping on page close if it is done dumping
you can now scroll the dumper's upper comment box
moved site links over to github
fixed a typo re downloading a file that is already in the db and adding tags at the same time
local tags delete fixed
tag parents now apply globally, like siblings
thumbnail resizer daemon is less verbose and more helpful on IOErrors
fixed namespace colours in manage tags dialog
open selection in a new page now works for single-file selections
upload tags progress bar misalign fixed
namespaced counts no longer total up in A/C writes
manage siblings dialog now catches collisions when you put in the old, not the pair, and asks you what you want to do
age predicate now supports hours
while search is not synchronised, A/C will now query db when it might have queried the current media
think I fixed a sometimes-not-initially-showing-first-frame-of-gifs problem
downloaders now run parsed tags through siblings and parents managers
maaaybe fixed a cache counts error
wrote a little more sibling and parent help, and updated schema

version 72

remote tag siblings!
remote tag parents!
simplified update nomenclature
streamlined client-to-server update objects and processing
streamlined server-to-client update objects and processing
did a bunch of work on content_updates
did some work on service_updates, too
improved client-side petition handling
a bunch of changes to how uploadpending works
the way pending downloads are stored is improved
mappings tables are collapsed server-side
file tables are collapsed server-side
moved tag siblings and parents to content_updates
moved file upload/download to content_updates
reworked tag content_updates to be more streamlined
addfile/tagrepositoryupdate is made entirely content_updates, now
simplification in the way deleted content is stored
repo update processing string is muchly improved
repo update processing is smoother!
a slight change to default behind-the-scenes searches
improved how the server creates updates on service init
improvements to service_identifiers_to_statuses_to_tags sub-init and related parts of tags_manager
some misc nomenclature improvements
fixed a denypetition bug for file repositories
fixed optimised petition handling in server
fixed posting tags in dumper
fixed a problem with getunknownaccount
server now analyses on vacuum, like client
fixed a lolbug in thumbnail resizer daemon that was continually resizing thumbs
fixed a bug in cache counts, which should fix the 'pending menu disappeared' problem
corrected num_archive count in file repos
downloading files is simpler and faster; download menu is simpler
fixed server initialisation
updated to new sqlite version

version 71

collapsed the four mappings tables into two tables
merged the two active_mappings tables into the mappings table
made a great number of changes to how mappings and active_mappings are stored and processed throughout
'active' and 'null' nomenclature is now 'combined'; null service_ids are now just ints
improved deletepending so it isn't so tough on the a/c cache
tags regex dialog entries 'for all files' and 'just for this file' was all broke
A/C read now bumps the exact match of the entry to the top of the list, if its count is non-zero
fixed A/C for weird-character queries, like '['
the dumper now makes success/error noises as appropriate
you can turn these noises off in the new sound tab in file->options
screwed around with garbage collection while checking mimetypes during pre-import
adding a tag parent will spam-add the actual parent tags to every child instance for the appropriate service
updated db diagrams
revised my sibling chain collapsing algorithm
locked db on init dialog message is improved a little
system:limit added to combined file service searches
num pending menu counts now split into with (pending/petitioned)
corrected a server db index oversight
newgrounds title tag fixed

version 70

tag parents db stuff
tag parents manager
tag parents predicate and matches indented expansion
and drawing it!
tag parents top result reordering
tag parents top result insertion
tag parents actually doing what they do
polished off tag parent help
made a full-search table to speed up tag A/C requests
added tag full-text search
reworked how tag and file services add and reset, to reduce A/C time
reworked recalc active mappings to be more beautiful, if not faster
fade animation timer improved in several ways
thumbnail fetch made much smoother
newgrounds artist downloading for games and movies added
newgrounds subs added
download panel input now highlights on init
4chan filename tag added
the setfocus on filter close is neater
age phrase is now 'imported [time] ago'
age is now shown in fullscreen
can now copy file from fullscreen, from menu or shortcut
4chan pass authentication improved
fixed a couple tag service precedence sync bugs

version 69

first dialog for tag siblings
tag siblings db table
tag siblings manager in controller
tag siblings display in tagsboxcpp
tag siblings display in tagsboxflat
tag siblings display in tagsboxmanage
tag siblings display in tagsboxactiveonly, via A/C dropdowns, for both read and write
top result sibling switcheroo in A/C write
A/C db fetch now does siblings too
db tag search does siblings
siblings help, with some nice mini-charts
I fixed collections, which were typo-broke; I'll make sure it doesn't happen again
more granular subs error handling, meaning individual file failures won't crash an entire sub
individual subs can now be paused
e621 fixed for real this time
A/C improvement that slows tag updates a little but should stop A/C lag after an update
as a result, trying out dropping the CPU-intensive fatten_ac_cache maintenance call
A/C read will now update system preds every time you click on it, so inbox/archive counts will stay accurate
A/C "all known files + tags" will no longer show the mega-laggy total file count
a list -> tuple convenience fix in sanelistctrl
if you don't have any pixiv credentials set up, you will now no longer get the option to start downloading pixiv stuff
fixed a tiny typo in the thumbnail resizer that made it wait far more politely than was intended
slight change to ratio system pred that fixes some lockups, sometimes, I think
fullscreen flash and video will get a pixel of whitespace on the right

version 68

fullscreen view now takes addmediaresult
export to zip!
import from zip, with tag regex stuff too!
export to encrypted zip!
import from encrypted zip!
import encryption help in advanced.html
import process code processes mimes less jankily
delete after successful import checkbox added
import regex tags dialog now has # namespace
import regex tags dialog collects its info a bit more intelligently
new confirm exit client option
subs will save their progress every 20 files downloaded, so pause/restart is less punishing
simplified thumbnail cache retrieval
some thumbnail waterfall and animation tuning, let me know what you think
e621 fixed
danbooru fixed
reworked booru gallery page num calc, which was completely screwed
DA fixed for most cases
booru thumb parsing made a _little_ more permissive
server now uses the same new file storage system as the client
download cancel buttons have a little better feel
new thumbnail prefetch thread makes thumbs load a load faster
fixed a little rubbish naming semantics in ClientGUI
reorganised and streamlined much of the encryption code
updated a bunch of help screenshots

version 67

subscription db access improved
subscription 'delete subs seemingly at random' bug fixed
some nice subs help
subs and repos, if changed during a pause, will restart with new changes
subs and repos will automatically pause while their respective dialogs are open
new namespace | regex listctrl in regex dialog, instead of old sctvcp rubbish
new /aa/aa...0 file storage system for client
fixed deleteorphans mime issue
export and copy files now export writeable files, not read-only
rejiggered daemon db access, improving maintenance reliablity
dumper and 'show in new page' pages now process content updates correctly

version 66

subscriptions done! works for all normal download types
pause repo sync, pause subs sync
all download sites moved to the new system
lots of small changes to how download code works
pixiv tags is renamed to pixiv tag, since it only does one!
fixed DA
fixed DA again!
fixed DA tag parsing for all the diff types of username re http://help.deviantart.com/106/
fixed pixiv, somewhat
fixed giphy
fixed boorus for gallery_advance_num > 1
boorus reset in db to default
fixed copy files
fixed getmime
a silly parent assignment for A/C meant they were not closing with pages
made the cpu burn for the thumbnail resizer a bit more polite on hdd
repo sync daemon and subs sync daemon combined
some timing adjustments in sync daemon
subscription_type constant -> site_download_type
fixed temp folder being cleared on startup (a read-only issue)
advancedoptions classes now support setinfo

version 65

added subscriptions dialog
added prototype subscription daemon, but not yet activated it
added downloader classes
began reorganisation of nearly all download code
moved parsing around
moved advancedhttpconnection around
moved serviceupdate around
moved some content_update stuff around
moved some constants around
files in client_files now have extensions
files in client_files are now read only
pdf launch is simplified as a result of this new ext stuff
fixed find similar images; it was just a typo
also fixed initial predicates string display
rejiggered the thumbnail resizer again; due to my stupidity, it was causing lag
thumbnail resizer burns a little more cpu when there are >10,000 or >100,000 thumbnails to render
it also now does thumbs in random order, for a couple of good reasons
upgraded sqlite, hopefully some queries will run faster?
upgraded to python 2.7.4. please report any weird errors

version 64

got rid of system:not_uploaded_to - now system:file service, which is a lot more powerful
some display changes to system predicates to make them a bit more human-readable
added export dialog
made a couple small changes to help about the export dialog
collect by rating
sort by rating now works for collections, using an estimate (only accurate when also collecting by that rating)
built a custom radiobox class and added it to ratings dialog so it has radio buttons again
new ratings filter for local_ratings_like services. works just like normal inbox filter
added a little info in help about this new filter
new session manager for pixiv and hentai foundry, reducing number of session inits the client has to do
new init welcoming window for first boot
new politer error message for when db is locked on boot
shift+arrow keys now pan in fullscreen mode. you can change them in options as normal, if you like
resize thumbnails daemon is now part of maintenance thread stuff. hopefully less blocking now
inbox pages will remove media when you archive stuff, again. this is just a bit I forgot to update from last week's predicate overhaul
fixed and changed the way hitting the change tag service button works
I fixed previews and fullscreen cache estimate in local options too!
fixed giphy so it works with their new api
rejiggered the update notification order to be a bit more accurate
updated auto-setup code and help to use new no-ip domain rather than old raw ip
v64 will also update the old ip to the new no-ip domain automatically
bit of variable renaming
cleaned up a couple menu memory leaks
updated a little of future.html

version 63

added include/exclude namespace predicate
added intelligent delay to A/C to smooth out db requests
rejiggered the A/C code so that spamming/pasting several characters and then deleting some will re-search for matches as appropriate
cut size down to just B, KB, MB, GB
maybe I fixed system:ratio in certain cases!
some general predicates display syntax changes
(archive/inbox) and (local/not local) mutual exclusivity with new system
system:num_words
right click -> open selection in new page
upgraded from PIL to Pillow, to zero noticeable effect lol
had another look at dodgy animated gif transparency; couldn't figure it out
CMYK jpegs images now supported! interlaced pngs simply not supported with PIL/Pillow yet
numpad delete added as a valid shortcut key
removed an old buggy deleteorphans line
new thumbnail resize daemon to fill out the thumbnails directory in the background
collections with only one file will now be converted to singletons
screwed around with locale settings. not sure if this will break things!
manage tags and ratings now in fullscreen rmb menu
on downloads, ( url -> hash ) pairs are stored in db, even if the file was already deleted or already in db
a few additions to help
reworked the spinctrls system preds to not have 100 as default max
changed a bunch more spinctrls, including memory sizes, in local options
and a statictext that wasn't showing number of thumbs for cache size
bugfix in ratings filter

version 62

remade a/c dropdown to a better, less buggy class
reworked a little of the a/c dropdown focus logic
reworked a little of the a/c dropdown show/hide logic
a 'sometimes hangs on "gui" on startup after crash' bug is fixed
can copy from listboxes with new right click menu
dumper page will protest if you try to close it while it is still dumping
import page won't protest if you try to close it while paused
num_words will show on pdf right click
fixed the network version mismatch exception, exactly one week late lol
couple of tiny menu memory leaks removed
couple of misc gui tidies
custom filter favourites save and delete buttons will grey out as appropriate
bit of general code cleanup
we nested classes now
moved contentupdate class from cc to hc, in prep for repo update rewrite and contentupdate gen

version 61

session db tables
session manager objects
session net code done
session failure recovery
registration key table
registration key requests
some registration help added
harmonisation of many http request names
extracted uploadpending code from db transaction, making it a lot more polite and architecturally correct
pixiv now parses the ascii creator name as well as the japanese counterpart. No option to choose between them yet!
improved processcontentupdate to support more actions, making db a little more serial
home and end now scroll to focussed media
some http timeout tweaks, to stop the 'can't connect to server' problems on complicated queries
a typo in modify account types was fixed
manage tags dialog shouldn't fritz out quite so much when a service doesn't have post_data permisisons

version 60

pixiv account management dialog
pixiv gallery parse
pixiv page tag parse, including creator and title
pixiv user download
pixiv tag download
collapsed the page chooser a little to fit pixiv in comfortably
pdf support!
pdf launching via preview/fullscreen button
pdf header parsing with num_words rough estimate
system:mime=application/pdf
system:mime=application
custom filter favourites done! with default, previous, save, save as and delete
some small changes to temp folder logic
vastly improved scroll-thumbnail-prefetch logic. important bits all rewritten and made sane. data and gui rows line up better, with fewer bugs
vastly improved thumbnail fade. now attempts to render at 60fps, and calculates alpha much more efficiently
thumbnails won't fade if they are off-screen; they'll just draw in one frame
fixed thumbnail last row drawing problem (again!)
due to daemon-db-spam, db maintenance check was only firing on pc wake from sleep, now it fires properly every twenty idle mins
maintenance timer won't fire directly after waking from a sleep
deleteorphans copes with alien files a bit more gracefully
A/C appearing on top is mostly fixed
some general A/C weirdness is fixed/made better
reworked some of the thumbnail selection and scrolling code to line up better
thumbnail window will stop spamming 'scroll to focus' every time something small changes. it should only do it on fullscreen close and non-ctrl or -shift mouse/key events now
some general focus code was looked at
ratings filter now supports the fullscreen_switch shortcut
just a couple changes to help to include the new 'f' fullscreen_switch shortcut
10s timeout on http queries
http redirect code is improved
http cookie handling code is improved
made hf session establishment a bit cleverer

version 59

new vastly improved 'collect by' dropdown
fullscreen 'f' shortcut
fullscreen default option
added tumblr prototype parser, with tags
server init and repository init typos are fixed
a misaligned sizer was fixed
a bunch of spinctrls no longer init at <= 100
the problem with the last thumbnail rows sometimes not drawing properly should be fixed
A/C lag calculates a little more intelligently
manage ratings for collections now works

version 58

new custom staticbox sizer/panel put in about fifty places
big layout changes in management panels
I made a new, better set of sizer flags for nested sizers
testing out several new borders
several other small layout changes, lining things up
a few colour changes with the new classes
advanced options class is better, more streamlined
advanced options class is less flickery, I think
A/C spazzes out less when dragged about
A/C queries db less annoyingly, especially when you type fast
the thumbnail display draws its whitespace a little better, filling in the right-hand and bottom-end gaps
importing thumbnails now add to the last row one thumb at a time (rather than at one row at a time) once the scrollbar appears
shift + f7 now sends things back to inbox, like a reverse archive.
this works from thumbnail menu
and in fullscreen
new menu item 'remove these' for thumbnails, anywhere, including dumper!
'tags just for this file' now works a looooot better! it unions all selected tags, and detects deselections better
giphy downloader added
giphy tags added, thanks to @fluffy_cub
copying file now _also_ copies paths to clipboard; depending on where you paste, you'll get the files or the text appropriately
new shortcut (default ctrl+m) for 'focus media'
focus search now works for manage tags and tags regex dialogs
think I worked out the last of the up/down arrow cursor shortcut problems
a ton more code cleaning
hentai foundry artist not found 404 error detected
in regex dialog, tags are cleaned before display on screen
4chan ban is now recognised, and thread dumps are hence suspended
when ordering tags by incidence, disputes are now resolved by lexicographic
fullscreen init is a little faster for the first image
select imageboard dialog now toggles tree with double click
select imageboard dialog is also taller

version 57

added quick vs accurate slider for numerical ratings filter
new slider will remember where you left it last
ratings filter internal comparison logic is more sane
'already rated' will now show the specific rating beside it
ratings filter (hopefully!) chooses better comparisons, in about four different and new ways
you can now ratings filter just one file
ratings filter code is neatened a whole lot, especially on my horrible variable names
I may have fixed an equality ratings filter bug
internal rating data -> pretty text for display is more flexible and accurate
going back several steps on ratings filter will no longer forget to re-rate already rated files
delete from dumper no longer screws with the indices
dumper is more stable, less likely to bail on the whole dump if something odd happens
dumper should recognise duplicate file errors and recover the fail
if the user has idled for the last twenty minutes, the db checks to see whether it can optimise itself
fixed flickering on fullscreen drag
fixed flickering onn fullscreen image init, meaning smoother mousewheel/whatever scrolling
fixed a graphical glitch in manage boorus dialog, and a bunch of others (removed all staticboxsizers)
fixed a similar glitch in manage ratings dialog - it now has dropdowns; I'll probably make it radio buttons again soon
fullscreen page navigation (previous, next, first, last) shortcuts are now customisable
custom filter now supports these new page nav shortcuts
can now do home/end and page up/down in shortcuts
shortcuts listctrls now sort by action by default
I improved the my custom listctrl's code generally
little less ugly code, here and there
fixed a whole ton of regular little bugs

version 56

new existing tag sort dropdown
an option to set default existing tag sort
dumper error logic improved
dumper now does random sort properly!
double buffering hilariousness
some reordering of the local options dialog
made a 'shortcut' entry object
made a parent frame object
collection thumbnails will redraw if the sort changes
add media and add thumbnail data and graphics logic is improved
4chan pass now authenticates for an entire year
bit more in help about the autocomplete tag entry
tag petition processing speed is a loooot faster

version 55

custom filter prototype ready!
added some info on custom filter to help (in advanced.html)
you can now set a default tag service in options
tag rexeg dialog's listctrl refreshes without scrolling up to the top every time
tag for this file in tag regex dialog applies to all selected, not just the first
scrolling in autocomplete entry will scroll services in manage tags and tags regex dialog
ctrl+scrolling in autocomplete entry will scroll through results
complete rewrite of image prefetch calculation
couple of content-update display bugs in ratings filter fixed
in manage ratings dialog, sliders will preset to best guess at current score
dumper should be fixed
resized active predicates box (much smaller)
a deprecated call updated
added some forum extensions

version 54

several new flash key and mouse event bugs fixed - now: mouse in = to flash | mouse out = to frame
the problem with scrollable management windows not drawing things on scroll is fixed; staticboxsizers now replaced with "- title -"
ratings filter now accepts several shortcut keys, directing to whichever window the mouse is over
the ratings filter windows will redraw themselves appropriately on content and service updates, e.g. changing tags with F3
ratings filter will choose the files to compare to more intelligently, both from the db and the currently-rated pool
the way services are identified behind the scenes is entirely overhauled and improved
the review service dialog is better as a result
all dialogs will now cancel-exit on pressing escape
started fade in and out for thumbnails
up cursor arrow in filter now skips
the tags regex dialog is refitted so it will refit on small screens better
dumper now dumps in its own thread (no longer locks up the gui while uploading)
the calculation of num_frames for flvs was off by a thousand, lol
synchro/wait shortcut typo fixed
inbox and other icons moved to background in fullscreen
fullscreen makes less db queries r.e. ratings
a couple managementpanel resize issues corrected
a couple drawing/flickering issues corrected
some updates to help
I made a forum to my site, and added links in the client and help

version 53

several terrible ratings filter bugs fixed
ratings filter can now rate files internally
ratings filter frame now has a statusbar
system:rating:service=uncertain added
upgraded to wx2.9, which included a ton of bug fixes
sort by namespace wasn't working due to an odd typo
collect by logic changed a little so that 'none' values are collected
the shutdown lag from v52 is corrected

version 52

ratings filter prototype is done
bit of help docs
more intelligent cpu burn on shutdown
dumper can manage flood time error, and recover from it
whole load of work done on mappings optimisation, meaning mappings updates, uploading mappings and approving petitions work about 100x faster!
forgetting pending uploads now throws up an ok/cancel dialog
had another look at overwriting deleted tags

version 51

dumper deals with failed uploads better, although there is more work to do.
flv support for client and file repo!
reworked tag services' permissions calculations on thumbnail menu
new fatten A/C cache on shutdown
system:untagged works again for null tag repo ('all known tags')
advanced import options now default no limit for size and resolution
ratings service now have a little info on the review services frame
new shortcut keys (ctrl-b and -n) for animated gif previous/next frame in fullscreen
added thoughts on plurals to tag schema help page
hentai foundry downloads now interleave pictures and scraps
removed db locks on file/thumbnail reads
you can open the page picker frame with the mouse by going view->pick a new page
the copy->whatever thumbnail menu is more concise
system:hash now works for null file repos ('all known files')
there is now a title: regex on the tag regex dialog
servers now store updates in a folder, not inside their dbs.
couple misc changes to help pages

version 50

4chan pass added
captcha timings in dump are better
dumper's file ordering is less insane
some general updates to regex tag dialog
new box on regex tag dialog for tags for single files
regex tag dialog now supports arrow key up and down to change repo
entirely new and vastly improved listctrl class
collect by is now a thousand times more complicated, but also much more powerful
the top-left media-flicker display bug in fullscreen is fixed
some logging improvements
some tag cleaning improvements
thumbnails have less cluttered tag headers
some small changes to help
fullscreen precache doesn't do gifs any more, which should make things smoother
got rid of the awful view pending uploads dialog. I'll improve it later
started remote ratings, but its not done yet

Revision #2
Created 12 March 2021 17:12:23 by CuddleBear
Updated 12 March 2021 18:24:07 by CuddleBear