We're looking for an outstanding Python programmer to help us develop data sharing platforms and software for neuroscience data. This is a large collaboration between ~50 neuroscientists from 20 labs around the world.
We published a new paper in Nature Neuroscience about our spike sorting method. This paper is accompanied by an open source software suite named klusta, which includes automatic and manual programs for extracting spikes from large multielectrode recordings.
In the research lab where I work, we've been developing a data processing pipeline for several years. This includes not only a program but also a new file format based on HDF5 for a specific type of data. While the choice of HDF5 was looking compelling on paper, we found many issues with it. Recently, despite the high costs, we decided to abandon this format in our software.
In this post, I'll describe what is HDF5 and what are the issues that made us move away from it.