Hello Yizhuang,
To answer your question “is 1000M the upper limit for Java memory”:
No, it depends on your RAM, Java version and operating system.
You can try higher values until the Java process fails to start.
I found that the largest alignments I could compute PCA for were approximately as follows:
With 1000M of Java memory:
500 rows x 5,500 columns (nucleotide)
500 rows x 2,500 columns (peptide) (larger score matrix)
With 8000M:
500 rows x 40,000 columns (nucleotide)
500 x 60,000 failed (out of memory)
With 10000M:
500 x 60,000 succeeded
This is for Mac OS/X with 16GB of RAM.
PCAs for large alignments will take a while!
On my MacBook Pro (2.8GHz):
-
500 x 30,000 (nucleotide) took 25 minutes
-
500 x 60,000 (nucleotide) took one hour
I hope this helps give you (and others) some idea as what size of PCA can be computed in what memory space.
In the longer term, we should like to provide a web service that could handle larger computations (and faster), however this is for the future!
Best wishes,
The University of Dundee is a registered Scottish Charity, No: SC015096
···
Mungo Carstairs
Jalview Computational Scientist
The Barton Group
Division of Computational Biology
School of Life Sciences
University of Dundee, Dundee, Scotland, UK.
www.jalview.org
www.compbio.dundee.ac.uk
From: Yizhuang Zhou zhouyizhuang3@163.com
Sent: 06 January 2017 09:43:26
To: Mungo Carstairs (Staff); jalview-discuss
Subject: Re: Re: [Jalview-discuss] out of memory
Thanks, Mungo.
Yes, I can successfully perform PCA for some small data when I changed it accourding to your suggestion. But I faced the same problem for larger data. It still did not work even when I set them to 5000m or 8000m. So is 1000m upper limit? If yes, how can I perform PCA for my large data?
Thanks in advance!
Best wishes,
Yizhuang Zhou
2017-01-06
Yizhuang Zhou
发件人： Mungo Carstairs (Staff)
发送时间： 2017-01-06 16:40:09
收件人： Yizhuang Zhou; jalview-discuss
抄送：
主题： Re: [Jalview-discuss] out of memory
Hi Yizhuang,
I think your default setting of max=268400000 is about 260MB, so less than the suggested 1000m.
Can you try commenting out the original defaults thus:
##lax.nl.java.option.java.heap.size.initial=33554432
##lax.nl.java.option.java.heap.size.max=268400000
and including the suggested
lax.nl.java.option.java.heap.size.max=1000m
lax.nl.java.option.java.heap.size.initial=500m
and let us know if this helps.
Thanks,
Mungo
Mungo Carstairs
Jalview Computational Scientist
The Barton Group
Division of Computational Biology
School of Life Sciences
University of Dundee, Dundee, Scotland, UK.
www.jalview.org
www.compbio.dundee.ac.uk
From: jalview-discuss-bounces@jalview.org jalview-discuss-bounces@jalview.org on behalf of Yizhuang Zhou zhouyizhuang3@163.com
Sent: 06 January 2017 07:20:58
To: jalview-discuss
Subject: [Jalview-discuss] out of memory
Hi,
I used Jalview to perform principle component analysis, but it said “Out of memory when calculating PCA!! See help files for increasing Java Virtual Machine memory”. Then I see help files (http://www.jalview.org/jvmmemoryparams.html). It told me to add two lines into Jalview.lax:
lax.nl.java.option.java.heap.size.max=1000m
lax.nl.java.option.java.heap.size.initial=500m
When I opened the file Jalview.lax. I found that there were already two lines which showed:
lax.nl.java.option.java.heap.size.initial=33554432
lax.nl.java.option.java.heap.size.max=268400000
So my question is whether we should add two lines to the end of this file? The default memory is larger than your suggested memory, if we change or add these two lines, can we fix
this probelm? Even we added these two lines, the problem was not fixed, it still said "Out of memory when calcu
lating PCA!! See help files for increasing Java Virtual Machine memory".So what
is the problem and how to fix it?
Thanks in advance!
Best wishes!
Yizhuang Zhou
2017-01-06
Yizhuang Zhou
The University of Dundee is a registered Scottish Charity, No: SC015096