Saving Varna states

Hi Yann - thanks for the reply!

I'm sorry I hadn't looked at it yet.
Shouldn't be too hard to do but I haven't had the time yet ( something about a collaborative book
chapter that has to be written jointly with not-so-responsive coauthors, ask Jim! :wink: )

ah yes. I'll send you another email about that in about 10 mins ( :slight_smile: ).

These are the features that we would really like to be able to restore:
-the current base positions in the diagram

ok, will do.

-the rotation of the structure

I prefer that we do not save this one, since a rotation will only affects the base coordinates.

Do you mean that the current base coordinates include the rotation ? If so, that's great - the idea is that if the user prepares a diagram 'just so', then they get it back when the load the project again.

-the draw algorithm

ok

-Annotations added to the sequence

Which one would you prioritize?
Textual annotations related to loops? of absolute XY coordinates?
Helices? Highlighted regions? Chemical probing style?

I'll leave it to Jan to work out the precise order, but I'd consider anything user created to be essential (any text annotations or interactively coloured parts of the structure for certain).

-base coloring

Single base styles then?

not sure. probably !

-zoom level

Do you really want that?
In my opinion, it might be preferable to "make everything fit" when loaded.

good point. However, the philosophy we took with Jmol was that if the user makes a special view, we don't mess with it when it gets reloaded - but provide some Jalview menu options above the Jmol window to allow the position to be reset, should the user want to do that.

-backbone color

ok

-base-pair style

ok, this will require exporting base-pair types (Leontis-Westhof classification)

How are those normally stored ? Jan and I have discussed the possibility of making the base pair representation within Jalview's datamodel richer - to facilitate visual analysis of the alignment w.r.t. base pairings, but we haven't identified the ultimate base pair datamodel as yet.

-color map

ok

-background color

ok

I also noticed that the selection of the sequence by a mouse click is
gone after one releases the mouse. In contrast to regions selected by
dragging a box around them. I'm not sure if it's by purpose by I think
it makes mouse selections a bit uncomfortable.

I see, good point!
So you would expect a single click on a base to persistently select the base, right?

I think that's the normal expectation - having watched other people use VARNA as well, they do expect that when a base starts to flash when clicked, that it should stay flashing. However, it may be that these users are used to the way that Jalview operates - where a single click selects a position.

Please note that you can use shift to toggle/add bases/boxes to the selection.

Another thing is that we would like to send sequence or structure edits
that where made in VARNA back to Jalview. Are there some public usable
Listeners for that? I could not figure them out.

Yes, we have a couple of listeners, but I'm not sure their design is clean enough to
be used within JalView. Could you again specify which features you had in mind?

We'd actually like to listen in to the following events:
* mouse over base position
- when the user moves the mouse on to a base, an event indicating the position within an associated VARNA RNA model is raised (reference to base, reference to model containing base).
* user selects part of or clears a selection on a structure
- event gives reference to model and list of bases selected, or null if the user cleared the selection.
* user modifies the RNA secondary structure
- event gives reference to model [and optionally - region of structure affected to allow efficient updating of Jalview's model].
Other events concerning colouring and annotation modifications could be used be Jalview in future versions - but at the moment, the only data exchanged between jalview and VARNA is sequence, base pairing, mouseover highlighting and (eventually) selections.

Do you think you could do something about this stuff?

Most certainly Yes, but it will depend on the time frame.

Jan is due to finish up his GSOC stint on 22nd August - which is almost certainly way too soon for you to do anything much. However, it would be nice if we could have interactive mouse overs/selections, and maybe structure edits exchanged between VARNA and Jalview by the end of his project - if that were at all possible.

Varna state saving is less urgent, but my hope is that Jan and I will be able to finish the VARNA integration ready for incorporation into Jalview series 2.7.x, and for this to happen, we need to be able to store VARNA states in Jalview projects.

I have started to lay the groundwork for VARNA state saving to save Jan some pain with XML schema modelling, and I'd suggest you start by working out a standard api for saving and loading these VARNA states that can import or export data (ideally via an abstract InputStream or OutputStream provided by the caller). Jan can then implement the necessary code in the VARNA/Jalview binding to call the API when a jalview project is saved/loaded. That way, integrating future updates to VARNA should require minimal (or no) modifications to the Jalview project save/loading code, and providing you have a basic format for representing the core aspects of a VARNA visualization that will be accepted by future versions of VARNA, you can embellish it at your leisure.

路路路

On 09/08/2011 09:13, Yann Ponty wrote:

Jan, Jim,

Do you mean that the current base coordinates include the rotation ? If
so, that's great - the idea is that if the user prepares a diagram 'just
so', then they get it back when the load the project again.

Yes, that is the way it is done by VARNA, although we also need to export the layout algorithm because it affects the way base-pairs are drawn (linear representation = arcs instead of lines) and some interactions.

-Annotations added to the sequence

Which one would you prioritize?
Textual annotations related to loops? of absolute XY coordinates?
Helices? Highlighted regions? Chemical probing style?

I'll leave it to Jan to work out the precise order, but I'd consider
anything user created to be essential (any text annotations or
interactively coloured parts of the structure for certain).

Yes, that is precisely the point, do you want:
   1) the annotations to be stored by JalView and sent/reflected by VARNA?
   In this setting, VARNA is only a view, and data can be passed from JalView to VARNA in real-time
   (i.e. annotations are part of JalView's model).
   2) the annotations to be drawn within VARNA?
   In this case, JalView relies on VARNA's annotations model.

It feels like 1) would be preferable, but would require listeners to keep JalView aware of user-edits.

-base coloring

Single base styles then?

not sure. probably !

-zoom level

Do you really want that?
In my opinion, it might be preferable to "make everything fit" when
loaded.

good point. However, the philosophy we took with Jmol was that if the
user makes a special view, we don't mess with it when it gets reloaded -
but provide some Jalview menu options above the Jmol window to allow the
position to be reset, should the user want to do that.

Sounds reasonable, although more crucial for 3D stuff (orientation is information) than for secondary structure schematics (except if you plan on having a 16s rRNA displayed within a 100x100 VARNA Panel :slight_smile: ).

-backbone color

ok

-base-pair style

ok, this will require exporting base-pair types (Leontis-Westhof
classification)

How are those normally stored ? Jan and I have discussed the
possibility of making the base pair representation within Jalview's
datamodel richer - to facilitate visual analysis of the alignment w.r.t.
base pairings, but we haven't identified the ultimate base pair
datamodel as yet.

I have been lazy with those (incremental design, so BP info is currently mixed with visual aspects/annotations), so do not read my code..!
As far as I am concerned (that is from a structure-centric point of view), a base-pair is:
   - a pair of nucleotides,
   - two interacting edges (Watson-Crick, Hoogsteen, or Sugar Edge) on the 5' and 3' sides
   - a glycosidic bond orientation (cis or trans).
You might want to take a look at the following seminal paper by Leontis/Westhof, introducing the nomenclature:
    Geometric nomenclature and classification of RNA base pairs - PubMed

I see, good point!
So you would expect a single click on a base to persistently select
the base, right?

I think that's the normal expectation - having watched other people use
VARNA as well, they do expect that when a base starts to flash when
clicked, that it should stay flashing. However, it may be that these
users are used to the way that Jalview operates - where a single click
selects a position.

We are clearly inconsistent on this one. Thanks for pointing it out, I just fixed this.

We'd actually like to listen in to the following events:
* mouse over base position
- when the user moves the mouse on to a base, an event indicating the
position within an associated VARNA RNA model is raised (reference to
base, reference to model containing base).
* user selects part of or clears a selection on a structure
- event gives reference to model and list of bases selected, or null if
the user cleared the selection.
* user modifies the RNA secondary structure
- event gives reference to model [and optionally - region of structure
affected to allow efficient updating of Jalview's model].
Other events concerning colouring and annotation modifications could be
used be Jalview in future versions - but at the moment, the only data
exchanged between jalview and VARNA is sequence, base pairing, mouseover
highlighting and (eventually) selections.

Ok, right now we do not have any mouseover monitoring (even within VARNA), but it shouldn't be too hard to do.

Do you think you could do something about this stuff?

Most certainly Yes, but it will depend on the time frame.

Jan is due to finish up his GSOC stint on 22nd August - which is almost
certainly way too soon for you to do anything much. However, it would be
nice if we could have interactive mouse overs/selections, and maybe
structure edits exchanged between VARNA and Jalview by the end of his
project - if that were at all possible.

Ok, I can prioritize that (especially given the new deadline for the VIZBI chapter :wink: ) and give you something to work with before the end of the GSOC

I have started to lay the groundwork for VARNA state saving to save Jan
some pain with XML schema modelling, and I'd suggest you start by
working out a standard api for saving and loading these VARNA states
that can import or export data (ideally via an abstract InputStream or
OutputStream provided by the caller). Jan can then implement the
necessary code in the VARNA/Jalview binding to call the API when a
jalview project is saved/loaded. That way, integrating future updates to
VARNA should require minimal (or no) modifications to the Jalview
project save/loading code, and providing you have a basic format for
representing the core aspects of a VARNA visualization that will be
accepted by future versions of VARNA, you can embellish it at your leisure.

Excellent suggestion!
Just out of curiosity:
Will JalView's "session" save format be a (zipped) bundle of files (with a bunch of specific readers on top of a ZipInputStream), or a unique merged (pseudo-)XML file?

Best,

Yann

路路路

_______________________________________________
Jalview-dev mailing list
Jalview-dev@jalview.org
http://www.compbio.dundee.ac.uk/mailman/listinfo/jalview-dev

--
Yann Ponty (mailto:yann.ponty@lix.polytechnique.fr)
CNRS junior research scientist-INRIA associate (AMIB team)
http://www.lix.polytechnique.fr/~ponty/
Laboratoire d'Informatique de l'Ecole Polytechnique (LIX)
Bioinformatics team
Tel: +33.1.69.33.41.12 fax: ++33.1.69.33.40.49
Address: LIX (L0), room 412-7
Ecole Polytechnique, 91128 Palaiseau,FRANCE

Yes, that is precisely the point, do you want:
  1) the annotations to be stored by JalView and sent/reflected by VARNA?
  In this setting, VARNA is only a view, and data can be passed from JalView to VARNA in real-time
  (i.e. annotations are part of JalView's model).
  2) the annotations to be drawn within VARNA?
  In this case, JalView relies on VARNA's annotations model.

It feels like 1) would be preferable, but would require listeners to keep JalView aware of user-edits.

this is definitely a hazy area - and one which Jan and I did not make much progress with. In principle, quite a lot of annotation can be stored within Jalview's datamodel and reflected in VARNA. There are even some generic key/value properties that VARNA could use to add in additional info like X,Y label position, etc. However, having embarked on a project like this in the past, I know its really quite tricky to get this kind of tight data integration, so for the moment, so I'd suggest we go with 2), and perhaps work towards something like 1) in the future.

i.e. make it so jalview can get annotation from VARNA and store it on the sequence/alignment, and can generate VARNA annotation from alignment/sequence annotation). Again, this is very much how we deal with Jmol : if the user does something in Jmol, then it gets stored in the Jmol state file; and when it's recovered, that annotation/rendering setting is only lost if they do something with the additional Jalview rendering or alignment options.

Sounds reasonable, although more crucial for 3D stuff (orientation is information) than for secondary structure schematics (except if you plan on having a 16s rRNA displayed within a 100x100 VARNA Panel :slight_smile: ).

yes - though what I was thinking of was that the user might have just a particular region in frame when they saved the project.

I have been lazy with those (incremental design, so BP info is currently mixed with visual aspects/annotations), so do not read my code..!

code ? what code 8^>

As far as I am concerned (that is from a structure-centric point of view), a base-pair is:
  - a pair of nucleotides,
  - two interacting edges (Watson-Crick, Hoogsteen, or Sugar Edge) on the 5' and 3' sides
  - a glycosidic bond orientation (cis or trans).
You might want to take a look at the following seminal paper by Leontis/Westhof, introducing the nomenclature:
   Geometric nomenclature and classification of RNA base pairs - PubMed

OK, will take a look at that sometime. However, just from what you say, it sounds like a base pair edge has twelve possible states (2*3 edges *2 for cis/trans) - definitely not something we can easily squeeze in to a single byte in a human readable fashion (I was thinking about this from the perspective of someone hand-coding annotation files for visualizing base pair contacts on an alignment).

users are used to the way that Jalview operates - where a single click
selects a position.

We are clearly inconsistent on this one. Thanks for pointing it out, I just fixed this.

thanks very much !

highlighting and (eventually) selections.

Ok, right now we do not have any mouseover monitoring (even within VARNA), but it shouldn't be too hard to do.

ace :slight_smile:

Jan is due to finish up his GSOC stint on 22nd August - which is almost
certainly way too soon for you to do anything much. However, it would be
nice if we could have interactive mouse overs/selections, and maybe
structure edits exchanged between VARNA and Jalview by the end of his
project - if that were at all possible.

Ok, I can prioritize that (especially given the new deadline for the VIZBI chapter :wink: ) and give you something to work with before the end of the GSOC

great! However, time is pretty short - Jan is supposed to finish up the majority of coding this week, leaving next week for documentation and polishing. So if it's more than a day or two's work, it might not be worth staying up all night to get it done :slight_smile:

Excellent suggestion!
Just out of curiosity:
Will JalView's "session" save format be a (zipped) bundle of files (with a bunch of specific readers on top of a ZipInputStream), or a unique merged (pseudo-)XML file?

Jalview's project files are zip archives containing jalview XML for each distinct alignment view, along with any additional dependent files referred to in the XML. Some kinds of data are simply embedded as UTF8 strings or base64 encoded blobs within the XML, but we keep PDB files as separate files in the archive to make compression more efficient, and also to make it easy for someone to extract them from the archive. I imagined that we might do the same for VARNA states, too - but either is possible. We could also cope with an api that just takes/generates a huge string, but having a reader/writer model is probably more memory efficient.

Jim.

路路路

On 09/08/2011 12:21, Yann Ponty wrote: