arrayMap feature update(s)
Over the last weeks, we have introduced a number of new search/ordering features to arrayMap. Some of those mimic functions previously implemented in Progenetix. Overall, the highlights are:
- ICD entity aggregation: all ICD-O entities with their according samples
- ICD locus aggregation: all tumor loci with their according samples
- Clinical group aggregation: clinical super-entities (e.g. “breast ca.”: all carcinoma types with locus breast) with their samples
- Publication aggregationall publication with samples in arrayMap
In contrast to Progenetix, we do not offer precomputed SCNA histograms. However, users can generate them on the fly, but should consider the specific challenges in doing so (e.g. noise background in frequency calculations).
Improved sample queries
The new database backend allows for better, real-time sample queries. Samples can be queried by specifying a number of parameters and/or keywords. The following example shows a text query for anything with “renal” in diagnosis text, ICD-O text or locus text, limited to platforms containing “affy” in the name, and having a minimum or 45000 probes. Also, there will be a limitation to samples having any change overlapping the DCKN2A locus - the selection is just being done:
After performing the search, the user is presented with selection lists containing parameters encountered in the current samples, for further exclusion options.
This description is now part of the new guide; pls. have a look there.
MongoDB database backend
We have recently moved both Progenetix and arrayMap from JSON flat file storage to a database backend driven by mongoDB. This improves data access speed and allows for real-time sample queries, among other things.
Updating the guide
I have killed the woefully inaccurate guide, and, never having liked the wiki system anyway, wrote a light-weight blog engine from scratch:
http://www.progenetix.org/cgi-bin/reader.cgi
Hopefully this will lead to a better documentation. Alas, you can always remind us!
Michael.
Change in data access policies
We have simplified our data access procedures. As of March 2012, for academic users registration is only necessary for
- any commercial use of the database
- maintaining private projects
- participating in collaborative studies containing unpublished material
We are happy about feedback and suggestions!
For any use of the site by for-profit entities, an individual license has to be obtained.
Enjoy,
Michael & the team at www.progenetix.org | www.arraymap.org
The arrayMap cancer genome resource manuscript http://arxiv.org/abs/1201.2677 is now available through arxiv.org. The manuscript has been submitted to PLoS ONE where it is currently being processed. Don’t forget the supplementary files. Enjoy!
Progenetix & arrayMap server hiccups …
With our servers on the move between facilities at the University of Zürich, there have been some DNS entry hiccups. If you didn’t memorize the previous IP addresses, you may have been out of luck for some hours … Hopefully settled, and I will try to accommodate for DNS server propagation delays in the future.
Improving array search with #MongoDB
After solely relying on structured flat file storage, we have made a first step towards a database back end implemented with #MongoDB. Currently, this is live only for the arrayMap array selection, which is now considerably faster. Feedback, please!
Progenetix/arrayMap JSON format update, and a bug fix
Master and temporary data files for arrayMap and Progenetix are saved in a JSON format, using Perl’s JSON::XS module. To speed up file I/O, we have deactivated the “pretty” option. This improves file I/O and size by approximately a factor of 2, but makes it rather unreadable. A simple way for reformatting the files (in Perl) would be:
my $json = JSON->new;
open INFILE, $infile;
open OUTFILE, “>”.$outfile;
print OUTFILE JSON::XS->new->pretty( 1 )->encode( $json->decode( <INFILE> ) );
close INFILE;
close OUTFILE;
Also, due to a comment by Felix Kokocinski, we removed a bug where wrongly formatted links (“http:/www. …”) lead to missing image files in the locus score function. The incomplete address is interpolated by Safari, but breaks in Firefox. Thanks!
Post with 1 note
Selection counters
A small interface change in the Progenetix/arrayMap pipeline now shows all case numbers in the feature selector. Enjoy!
