[Shotwell] Enhansment to shotwell

Fri Apr 23 20:17:57 PDT 2010

Ok, keeping my fingers crossed it will work fine for me (and a lot of
other parents:) )
Only have 35,000 or so... and counting...

Perhaps add so we can tag by just using keyboard...
Something like F-Spot... Works very well.

As soon as Shotwell can import XMP (f-spot/Digikam) and IPTC tags, then I
am sure a lot of people will be happy to try it out... :)

Looking forward to this...

/Bengt

Den Lö, 2010-04-24, 10:31 skrev Jim Nelson:
> Shotwell's not designed for that kind of scalability (yet!)  I'm more
> concerned with 10,000 photo collections.
>
> Right now I would want to design this correctly and cleanly, then do
> performance testing to determine if there are bottlenecks and correct
> them.  As I mentioned, we could design it so each Tag maintains its
> photos and its childrens' photos, adding and removing them
> dynamically.
>
> -- Jim
>
> On Fri, Apr 23, 2010 at 4:33 PM, Bengt Thuree <bengt at thuree.com> wrote:
>> Will the performance be ok with a photo collection of 50,000 - 1,000,000
>> Photos or above?
>>
>> Bengt
>>
>> On Fri, 2010-04-23 at 11:54 -0700, Jim Nelson wrote:
>>> So far, this sounds like steps in the right direction.  Some notes:
>>>
>>> Our general strategy in Shotwell is to load the entire database in one
>>> swoop at startup and maintain all database rows in memory.  Once the
>>> application is started, database access is limited to updates and
>>> deletes.  (There are exceptions, particularly in Event.)  Generally
>>> it's faster to do searching in memory than via the database.  It's a
>>> trade-off, but one we take for performance reasons.
>>>
>>> If you read the architecture overview on Data Structures, I talk about
>>> the strategy I use for signalling (i.e. listeners, or observers as I
>>> call them).  We definitely want each Tag to signal when it's children
>>> and parent is altered.  I would also recommend having these signals
>>> reflected by TagSourceCollection, that way an observer can register
>>> once than register for every Tag.  For example, LibraryWindow could
>>> register once and receive notifications on all Tags being moved
>>> around, updating the Sidebar accordingly.
>>>
>>> Also note that we've invested a lot of code time in fully synchronized
>>> data structures for the "big" objects and their collections: Tag,
>>> Photo, Event, etc.  Since each Tag maintains a ViewCollection of Photo
>>> objects, it's easy to recurse up and down the hierarchy gathering a
>>> HashSet of Photos.  It could be as simple as this:
>>>
>>> void get_photos_and_children_photos(Gee.HashSet<LibraryPhoto> photos) {
>>>     photos.add_all(get_view().get_all());
>>>     foreach (Tag child in children)
>>>         child.get_photos_and_children_photos(photos);
>>> }
>>>
>>> Since I don't expect Tag hierarchies to be crazy-deep, this is
>>> probably sound.  (There can be no loops in the tree, of course.
>>> You'll need to protect against that.)
>>>
>>> Alternately, each Tag could monitor additions and deletions of its
>>> children and maintain a separate ViewCollection of its Photos and its
>>> descendants' photos.  I'm not convinced that there's a big win here,
>>> but it's possible with the architecture in place.
>>>
>>> In short, I would discourage looking to the database for anything more
>>> than data persistence and organization.  My feeling right now is, this
>>> can all be done in memory faster and and more efficiently than via
>>> SQL.
>>>
>>> -- Jim
>>>
>>> On Fri, Apr 23, 2010 at 5:20 AM, Ruslan A. Bondar <fsat at proton-sss.ru>
>>> wrote:
>>> > -----BEGIN PGP SIGNED MESSAGE-----
>>> > Hash: SHA1
>>> >
>>> > Hello, Adam.
>>> >
>>> > I think the code change will contain following steps:
>>> > 1. Add field "ParentTagId" to TagTable
>>> > 2. Change the TagTable class to handle this field.
>>> > 3. Change the find_parent_marker and add_tag_page methods in
>>> LibraryWindow to provide recursive tree creation for tag tree.
>>> > 4. Change drag'n'drop handler to allow moving tags between branches.
>>> >
>>> > And something more, such as listeners of tag tree alteration etc. I
>>> think these changes will not affect any other functionality. And no
>>> major changes in design are required.
>>> >
>>> > I think it would be better to store only photos "owned" by this tag,
>>> but on load build the full list, containing photos from this and all
>>> child tags. And for now I see several ways:
>>> > 1. Create helper table to speedup loading, but this will slow down
>>> tag reassigning (Tag moving between branches). This table may contain
>>> only 2 fields: TagId and AllTagPhotos.
>>> > 2. Walk over the table recursively and build the list via
>>> concatenation, this will be slightly slower.
>>> >
>>> > These are the common thoughts. I'll dig the source deeper during the
>>> weekend.
>>> >
>>> > On Thu, 22 Apr 2010 19:25:51 -0700
>>> > Adam Dingle <adam at yorba.org> wrote:
>>> >
>>> >> Ruslan,
>>> >>
>>> >> thanks for your email.  As you probably know, we're also interested
>>> >> in hierarchical tagging.
>>> >>
>>> >> I think at this point we'd like to implement a single-parent system,
>>> >> in which each tag can have multiple children but only one parent.
>>> >> This is ticket 1401.  After that, we might consider extending the
>>> >> system to support multiple parents in some way.
>>> >>
>>> >> A single-parent tag system in Shotwell would work like this.  In the
>>> >> sidebar, the user should be able to drag any tag A onto any other
>>> tag
>>> >> B; then A becomes a child of B.  If the user drags any tag onto the
>>> >> Tags heading, then it becomes a top-level tag.
>>> >>
>>> >> If the user selects any tag, then Shotwell should display all photos
>>> >> which belong to the tag or any of its children.  The set of photos
>>> is
>>> >> sorted according to the View->Sort Photos settings.  If a photo
>>> >> belongs to multiple child tags, it is displayed only once in the
>>> >> sorted set.
>>> >>
>>> >> All tag names must be globally unique.  For example, I can't create
>>> a
>>> >> tag named "washington" under a parent "states", and another tag also
>>> >> named "washington" under a parent tag "cities".
>>> >>
>>> >> You said you might like to participate in coding this feature - that
>>> >> would be great.  If the description above sounds reasonable to you,
>>> >> as a next step maybe you could look at the Shotwell code and send a
>>> >> brief design proposal to this mailing list.   In your proposal,
>>> >> please describe the database schema changes you intend, plus any new
>>> >> major classes introduced (if any) and any major changes to existing
>>> >> classes. We'd be happy to comment on your implementation plan.
>>> >> Thanks!
>>> >>
>>> >> adam
>>> >>
>>> >> Ruslan A. Bondar wrote:
>>> >> > -----BEGIN PGP SIGNED MESSAGE-----
>>> >> > Hash: SHA1
>>> >> >
>>> >> > Hello all.
>>> >> >
>>> >> > I have an idea, maybe similar to
>>> http://trac.yorba.org/ticket/1401,
>>> >> > but I'll tell in more details.
>>> >> >
>>> >> > It would be great to allow organizing tags into graph.
>>> >> > So one tag may have multiple parent and multiple children tags.
>>> And
>>> >> > each of them may be assigned to the photo. It will be more
>>> flexible
>>> >> > to organize your photos. As an owner of large, enough, photo album
>>> >> > (~100 GB of photos) I can say - This feature is VERY useful.
>>> >> >
>>> >> > Also I'd like to participate in coding of this feature.
>>> >> > -----BEGIN PGP SIGNATURE-----
>>> >> > Version: GnuPG v2.0.14 (GNU/Linux)
>>> >> >
>>> >> > iEYEARECAAYFAkvNx+QACgkQRadsTlsSykPASgCcDBrgeobgyNweH2CLTaFt1xZu
>>> >> > gqUAn3jfVCjMrDPuwQJQWvh3KBYqdq7q
>>> >> > =4KsN
>>> >> > -----END PGP SIGNATURE-----
>>> >> > _______________________________________________
>>> >> > Shotwell mailing list
>>> >> > Shotwell at lists.yorba.org
>>> >> > http://lists.yorba.org/cgi-bin/mailman/listinfo/shotwell
>>> >> >
>>> >> >
>>> >>
>>> >
>>> > -----BEGIN PGP SIGNATURE-----
>>> > Version: GnuPG v2.0.14 (GNU/Linux)
>>> >
>>> > iEYEARECAAYFAkvRkIYACgkQRadsTlsSykNC4gCeJ6XCVc9RzUwluOwj/lo960xg
>>> > G30AnAvyDgsu+DK9raFT2LUZFOtf3APZ
>>> > =LAuc
>>> > -----END PGP SIGNATURE-----
>>> > _______________________________________________
>>> > Shotwell mailing list
>>> > Shotwell at lists.yorba.org
>>> > http://lists.yorba.org/cgi-bin/mailman/listinfo/shotwell
>>> >
>>> _______________________________________________
>>> Shotwell mailing list
>>> Shotwell at lists.yorba.org
>>> http://lists.yorba.org/cgi-bin/mailman/listinfo/shotwell
>>>
>>
>>
>>
>
>

-- 
Bengt Thuree   bengt at thuree.com