Keep the tree window hidden

Dear all,

I'm wondering if it is possible to configure the tree window so that it can be opened by user selection (menu/button) and no by default when showing the alignment.

Thank you,
Paolo Di Tommaso

Hello there, Paolo.

···

On 10/01/2012 13:00, Paolo Di Tommaso wrote:

I'm wondering if it is possible to configure the tree window so that it can be opened by user selection (menu/button) and no by default when showing the alignment.

Jalview doesn't provide any special option for this, but it would be fairly easy to do. I thought it might be possible with the existing API, but that also needs tweaking, so I've opened a feature request here:
http://issues.jalview.org/browse/JAL-1039

Jim.

Hi Paolo - just cc'ing the dev list so we all have a record of the emails!

T-Coffee can produce the alignment score in a text file, like this

http://tcoffee.crg.cat/data/79609a4a/sh3.score_ascii

OK.

Merging this information with the resulting sequence alignment, should not be too difficult to visualize the resulting color scheme in Jalview.

yes.

In my opinion there are two main use cases:

1) The users (or the developer in the case of the applet) provide the sequences alignment and the score ascii files as produced by T-Coffee. Jalview manage to convert the scores in its internal structures and render the alignment using the associated color scheme.

2) The user submits a T-Coffee alignment request. The web service send back the score ascii other than the alignment fasta file, and then the case fallback to the previous one.

At the beginning I would focus on the most simple scenario (which by accident is the one on which I'm more interested :)), I mean the Jalview applet version and make it able to render the colored T-Coffee alignment, adding an extra parameter to specify the generated score_ascii to the applet.

What do you think about that?

I think it sounds great. The TCoffee scores fall into a similar class as JNet scores - which get turned into alignment annotation and can be specified separately to the alignment. Extending the web service will require a bit more familiarity with Jalview's code, so I'd definitely suggest #1 is the place to start.

Jim.

···

On 30/03/2012 10:35, Paolo Di Tommaso wrote:

Hi Paolo.

I think easter holiday it will a good timeframe to start implementing the T-Coffee score rendering :wink:

great !

Is this the source repository for Jalview http://source.jalview.org/git/jalview.git isn’t it ?

yep. However, you can also work with https://source.jalview.org/git/jalview.git with your issues.jalview.org username and password.

The version of Jalview code to work from is the develop branch - this is the one that will become the next release. If you clone the repository and create a new local branch off that branch, you should be able to rebase your work on top of mine as I commit on to the development branch. This is the ‘gitflow’ model - where every feature is developed in its own branch that - if local to you - can be rebased on or otherwise merged with any incoming changes on the parent branch. When the feature is ready to be released, its branch is then merged onto the develop or release branch where it is needed. (there’s more info and a link about gitflow in this email http://www.jalview.org/pipermail/jalview-dev/2012-January/000205.html).

Moreover could you provide me some starting tips, based on we said in the previous days (for example which are the main classes on which I could work, the entry point class, how to setup a test environment, etc)

Your first step should be to checkout the source and see if you can get it to build… in principle, if you work in eclipse, it should import as a java project and building should ‘just work’ - but there is a little wrinkle: if the applet classes are not compiling due to a missing class, then you need to add the ‘plugin.jar’ library that sits inside your JRE’s lib directory to the project’s build classpath.
[ as an aside, if you use netbeans - there a netbeans project in the directory (nbbuild.xml) but I suspect it might have a few out of date dependencies that need renaming to the latest jar in the lib/ directory].

The main entry point for the applet is the one that is called in the applet tag - jalview.bin.JalviewLite. You should be able to set up an applet launch configuration (or appletviewer page) that loads an alignment, you can then add in the new parameter that specifies the t-coffee score file. The ant script that comes with jalview includes everything needed to make the jalviewApplet.jar if you want to package the applet up. In particular, ‘pubapplet’ will make a copy of the ‘examples’ directory under the build directory (called ‘dist’ by default) and put a copy of the applet jar there. You can then point a web browser at a page in that directory to test the applet (note - if you need to do any ajax stuff, you should start a jetty web server using the jetty maven module following the instructions here: http://maven.apache.org/plugins/maven-war-plugin/examples/rapid-testing-jetty6-plugin.html ).

The general series of implementation steps will be something like:

  1. create a new class in the jalview.io package to read the t-coffee score matrix for the alignment positions. This would be called in jalview.bin.JalviewLite after the alignment was loaded in order to add the annotation.
  2. create a new colourscheme in jalview.schemes for the T-coffee score colourscheme. These are called by the jalview.appletgui.SequenceRenderer class to render the visible portions of the sequence.
  3. make sure the colourscheme is added to the colours menu whenever t-coffee scores are available.

Initially, I’d take a look at the execution path from jalview.bin.JalviewLite to when a jalview.appletgui.AlignFrame is created and displayed, and try to understand how the different colourschemes are called by the SequenceRenderer class.

Once you’ve done that, we can talk about the best way to implement this colourscheme. The hardest bit is implementing an efficient lookup to find the t-coffee score for a particular position in a sequence, but for a first attempt, I shouldn’t worry about this. The easiest path will be to store the T-coffee scores for each sequence as a jalview.datamodel.AlignmentAnnotation object - which can be associated with a SequenceI reference. The ColourScheme implementation should then find the colour for a position for a sequence in the alignment according to the associated AlignmentAnnotation object for a sequence.

The final nuance for the colourscheme is to colour the sequence IDs according to the per-sequence T-Coffee score. Currently, Jalview has no support for shading sequence IDs according to some value, but there is support for colouring sequence IDs as specified in a jalview.datamodel.SequenceGroup (which are created from alignment annotation files and when the user clicks in a tree to subdivide the alignment).

All this should keep you busy over Easter ! Feel free to create subtasks on issues.jalview.org to help document your progress. I’ve started some here: http://issues.jalview.org/browse/JAL-1065

Remember, if you include the JIRA issue number for a task in a commit (e.g. JAL-1065 for this feature request) then the commits will be visible when they are pushed to source.jalview.org.

Jim.

ps. I remembered this morning that you originally emailed the list about making it possible to show the associated tree for an alignment by selecting a menu item like ‘show tree’ - http://issues.jalview.org/browse/JAL-1039
… you’ll probably think of a few ways to do that as you look through the Jalview code.

Hi Paolo.

So I wrote the T-Coffee score file reader and I was able to add a new MenuItem in the applet color menu.

great !

I wrote also a TCoffeeColorScheme BUT I stuck in this problem: I was assuming that the ColourSchemeI interface had a method returning the color for the sequence i and position j.

But this is not the case. It defines the following two methods :

public Color findColour(char c);

public Color findColour(char c, int j);

this is exactly the problem I expected you to hit - and is the main reason why this implementation needs a bit of thought. The simplest approach is to add an additional parameter specifying the sequence (SequenceI) which is being coloured.

Which is the best approach in your opinion to implement the color scheme ?

I need to think about that for a bit. For a first attempt, try the hack above and see what complications emerge. You could maybe consider extending the colourscheme interface with the overloaded method. Have a play yourself and see if you can get something working…

Jim.

···

On 05/04/2012 00:19, Paolo Di Tommaso wrote:

Hi Paolo.

We got it! Here it is a screen shot rendering the T-Coffee score color
scheme.

http://dl.dropbox.com/u/376524/jalview-tcoffee-scores.png

great ! So the only thing left is the ID colouring ? I've taken a look and added another task : http://issues.jalview.org/browse/JAL-1065

There will be a few other things to think about. Documentation is one, since you've modified the GUI (these are done via JavaHelp pages in help/html) and added a new applet parameter (which is documented in examples/appletParameters.html). The steps after that would be to provide a means for the user to load their own T-Coffee scores onto an alignment in the jalview desktop or applet, and ensuring the t-coffee colourscheme and scores can be saved and loaded again via the Jalview project (much trickier, so maybe not for this Easter - unless you're up for it!).

Jim.

···

On Thu Apr 5 23:54:32 2012, Paolo Di Tommaso wrote:

Hi Jim,

What do you mean for ID coloring? Could you provide a practical example?

I will update as well the documentation. About the other steps, before I would commit a stable version of what I've done, so you can provide some feedback on that.

Can I commit in the "master" branch, from where I've download the source ?

I more thing, how do you report error messages in Jalview? do you a particular method? syso.println ?

Cheers,
Paolo

···

On Apr 7, 2012, at 9:46 AM, Jim Procter wrote:

Hi Paolo.

On Thu Apr 5 23:54:32 2012, Paolo Di Tommaso wrote:

We got it! Here it is a screen shot rendering the T-Coffee score color
scheme.

http://dl.dropbox.com/u/376524/jalview-tcoffee-scores.png

great ! So the only thing left is the ID colouring ? I've taken a
look and added another task : http://issues.jalview.org/browse/JAL-1065

There will be a few other things to think about. Documentation is one,
since you've modified the GUI (these are done via JavaHelp pages in
help/html) and added a new applet parameter (which is documented in
examples/appletParameters.html). The steps after that would be to
provide a means for the user to load their own T-Coffee scores onto an
alignment in the jalview desktop or applet, and ensuring the t-coffee
colourscheme and scores can be saved and loaded again via the Jalview
project (much trickier, so maybe not for this Easter - unless you're up
for it!).

Jim.

Hello Paolo.

I see quite clear. I will give a look how it could be implemented.

ace.

It would the nice also to render somehow the msa consensus scores. I mean the last colored row you see in each block in the T-Coffee html result, identified by with sequence id 'cons'. Ideally it should be rendered as a colored line under the MSA redered by Jalview. This would be a third new feature.

there are two ways you can do this.
The easy one is to simply add the consensus scores as a new histogram style plot. You can see examples of code that does this in jalview.io.AnnotationFile, and JPredAnnotationMaker (which is a bit more complex).

I use Eclipse, but for Git I'm more confident with the cmdline tool. Would not be easier that I create a new local branch and then I push it to the remote repository. Like explained here http://goo.gl/fpkzU

What do you think?

yep - that's the safer way - sorry - couldn't remember if you were doing everything with egit or not. I was actually pretty impressed with egit when I took a look today, it's almost as good as gitx now.. but still way slower.

  I more thing, how do you report error messages in Jalview? do you a particular method? syso.println ?

Depends on the error and also where it occurs. The basic philosophy is
to throw 'Error' objects for implementation problems, throw exceptions
which get dumped to System.err if there are unexpected errors due to
input, and try to give an informative message to the user.

For alignment input files, there is a standard method for raising
dialogs for the user, but there isn't anything yet for importing
annotation files (something for my TODO list), so for the moment, send
errors to System.err and I'll take a look and see if there's a better way.

OK!

glad that made sense :wink:

happy Easter!
Jim.

···

On 07/04/2012 12:13, Paolo Di Tommaso wrote:

Hi Jim,

I've just pushed the changes integrating the T-Coffee score in Jalview.

So, what's new:
- A T-Coffee score file parser.
- A new menu entry in the color menu "T-Coffee scores" both in the applet and desktop versions.
- A new 'scoreFile' parameter in the applet to load the T-Coffee score file when including jalview into a web page.
- A new menu entry in the file menu, named "Load T-Coffee scores", in the desktop version (maybe it would have sense to add also to the applet).
- Updated the appletParameters.html page.

What is missing:
- JAL-1068
- T-Coffee consens score histogram
- Updating the manual for the remaining features

The new branch is named: origin/Tcoffee_JAL-1065

I will wait for your comment before continue with other developments

Cheers,
Paolo

···

On Apr 8, 2012, at 1:53 AM, Jim Procter wrote:

Hello Paolo.

On 07/04/2012 12:13, Paolo Di Tommaso wrote:

I see quite clear. I will give a look how it could be implemented.

ace.

It would the nice also to render somehow the msa consensus scores. I
mean the last colored row you see in each block in the T-Coffee html
result, identified by with sequence id 'cons'. Ideally it should be
rendered as a colored line under the MSA redered by Jalview. This
would be a third new feature.

there are two ways you can do this.
The easy one is to simply add the consensus scores as a new histogram
style plot. You can see examples of code that does this in
jalview.io.AnnotationFile, and JPredAnnotationMaker (which is a bit more
complex).

I use Eclipse, but for Git I'm more confident with the cmdline tool. Would not be easier that I create a new local branch and then I push it to the remote repository. Like explained here http://goo.gl/fpkzU

What do you think?

yep - that's the safer way - sorry - couldn't remember if you were doing
everything with egit or not. I was actually pretty impressed with egit
when I took a look today, it's almost as good as gitx now.. but still
way slower.

I more thing, how do you report error messages in Jalview? do you a particular method? syso.println ?

Depends on the error and also where it occurs. The basic philosophy is
to throw 'Error' objects for implementation problems, throw exceptions
which get dumped to System.err if there are unexpected errors due to
input, and try to give an informative message to the user.

For alignment input files, there is a standard method for raising
dialogs for the user, but there isn't anything yet for importing
annotation files (something for my TODO list), so for the moment, send
errors to System.err and I'll take a look and see if there's a better way.

OK!

glad that made sense :wink:

happy Easter!
Jim.

Hi Paolo

I've just pushed the changes integrating the T-Coffee score in Jalview.

I took a look - it works pretty nicely.

So, what's new:
- A T-Coffee score file parser.
- A new menu entry in the color menu "T-Coffee scores" both in the applet and desktop versions.
- A new 'scoreFile' parameter in the applet to load the T-Coffee score file when including jalview into a web page.
- A new menu entry in the file menu, named "Load T-Coffee scores", in the desktop version (maybe it would have sense to add also to the applet).
- Updated the appletParameters.html page.

OK - I'll raise a few issues on JIRA over the next few hours - mostly to do with reuse of existing code for doing things like resolving data URIs (its a bit of a pain in the applet to cover all bases and support legacy behaviour - so I might fix that patch). Your implementation also raises an interesting design question about attaching invariant indexes to sequences (as you do to index into the alignment scores matrix).

What is missing:
- JAL-1068
- T-Coffee consens score histogram
- Updating the manual for the remaining features

OK - will also put some more comments on the bugs.

The new branch is named: origin/Tcoffee_JAL-1065

I'll start to merge this onto the develop branch, since that code is in flux. My initial experiments seem to suggest it should all be fine, but more testing will be needed.

I will wait for your comment before continue with other developments

Thanks for taking the time to do this ! I also note that you have bravely checked in a directory of JUnit tests! This has forced me to think a bit about how to retrofit the legacy test data to this model. Most of our test data sits in the examples directory, so that we can get people to run import tests on their live Jalview installation if they run into problems (this is a nice way of checking that their local environment is good). Wonder if Groovy is the way to go here for a test harness ?

Jim.

···

On Tue Apr 10 13:19:27 2012, Paolo Di Tommaso wrote:

Hi Jim,

Your implementation also
raises an interesting design question about attaching invariant indexes
to sequences (as you do to index into the alignment scores matrix).

Well, it was the first option. But actually it may result to place unwanted constraints in future development.

It could be easily replaced using the sequence ID in place of the sequence index (assuming that the method SequenceI#getName() is retuning the sequence ID, isn't it ?)

What about this?

Thanks for taking the time to do this ! I also note that you have
bravely checked in a directory of JUnit tests! This has forced me to
think a bit about how to retrofit the legacy test data to this model.
Most of our test data sits in the examples directory, so that we can
get people to run import tests on their live Jalview installation if
they run into problems (this is a nice way of checking that their local
environment is good). Wonder if Groovy is the way to go here for a test
harness ?

Jim.

Yes. In a recent project, I've started using Groovy + Spock (http://code.google.com/p/spock/) and it's really a perfect match, above all if there are data based tests, like you are arguing. I think it could be really a good choice for Jalview test harness.

Best,
Paolo