{"componentChunkName":"component---src-templates-blog-post-js","path":"/brightfield_challenge_data_manifest/","result":{"data":{"site":{"siteMetadata":{"title":"tigue.com"}},"markdownRemark":{"id":"5fdf0863-1a8f-5e5a-97f7-869bf41a4d35","excerpt":"This Jupyter notebook (read: blog post) provides a high level overview of the files in The Allen Institute’s Brightfield Auto-Reconstruction Challenge dataset…","html":"<img src=\"http://reconstrue.com/projects/brightfield_neurons/demo_images/651806289_minip_turbo_banner.png\" width=\"100%\">\n<p>This Jupyter notebook (read: blog post) provides a high level overview of the files in The Allen Institute’s <a href=\"http://localhost:8000/brightfield_reconstruction_challenge/\">Brightfield Auto-Reconstruction Challenge</a> dataset. The dataset is about 2.5 terabyte of data. This notebook looks at the dataset from a file system level. To this notebook the dataset is just files, not image stacks.</p>\n<p>A <a href=\"http://reconstrue.com/projects/brightfield_neurons/challenge_dataset/specimens_manifest.json\">JSON manifest file</a> is produced, which is used as a convenience by other notebooks in this project.</p>\n<p>For more context see the project’s main notebook on Colab, <a href=\"https://colab.research.google.com/drive/1qvwT-SxHpZSLQ88VeIOkR296pbZfQqTK\">brightfield neuron reconstruction challenge.ipynb</a>.</p>\n<h2>Overview of files</h2>\n<p>This doesn’t get into higher level domain specific stuff (i.e. the main goal of innovating actual ML-assisted microscopy) rather the topic here is to take stock of the files and partition them into useful subsets small enough that compute can happen on Colab (answer: download and process all of one specimen’s files at a time, but only one at a time).</p>\n<p>Creating a manifest of the dataet is something that needs to (theoretically) be done only once. Programmatically walking the bucket is just an annoyance; it’s quicker/easier to just load a pre-built manifest. Later other notebooks, e.g. <a href=\"https://colab.research.google.com/drive/1ZxzDwD1UdYqhuTxckPLiOUYHin0MZmFQ#scrollTo=TaREcuFG6SSQ\">initial<em>dataset</em>visualization.ipynb</a>, will simply read the file <code class=\"language-text\">specimens_manifest.json</code> to know what files are in the dataset.</p>\n<p>actually visualizes the dataset on a digital microscopy level (read: show images), by deep diving on a single specimen cell’s data (image stack and SWC skeleton).</p>\n<h2>Access info</h2>\n<p>The challenge dataset is hosted on Wasabi Cloud Storage, which mimics the APIs of AWS S3 so all the regular ways of accessing data on S3 can be used to access the data</p>\n<ul>\n<li>Service endpoint address: s3.wasabisys.com</li>\n<li>Storage region: us-west-1</li>\n<li>bucket name: brightfield-auto-reconstruction-competition  </li>\n</ul>\n<h2>Overview of bucket’s contents</h2>\n<p>There are two parts to the data</p>\n<ol>\n<li>Training data (105 neurons, with manual SWCs): 2.2 TB</li>\n<li>Test data (10 neurons, no SWCs): 261.3 GB</li>\n</ol>\n<p>Each neuron is in its own folder off the root of the bucket. So the are over 100 folders with names like <code class=\"language-text\">647225829</code>, <code class=\"language-text\">767485082</code>, and <code class=\"language-text\">861519869</code>.</p>\n<p>Each neuron’s data is in a separate folder. Each folder consists of</p>\n<ul>\n<li>the input: a few hundred TIFF image files</li>\n<li>the output: one SWC manually traced skeleton file</li>\n</ul>\n<p>There is one unusual sub-root folder, <code class=\"language-text\">TEST_DATA_SET</code>, which contains the data for the ten neurons used during the challenge’s evaluation phase. These ten neuron image stacks <em>DO NOT</em> have SWC files.</p>\n<p>The goal is that some software will read the image stack  and auto reconstruct the SWC, without a human having to manually craft a SWC skeleton file (or at least minimize the human input time).</p>\n<p>So, the idea is a two phase challenge: first train with answers (SWC files), then submit 10 SWC files the program generates on the ten neurons in <code class=\"language-text\">TEST_DATA_SET</code>. </p>\n<p>sfirst train a auto reconstruction program using the roughly 100 neurons in the training data set, and check your results against the human traced SWC skeletons that each neuron’s image stack comes with. Then for the evaluation phase</p>\n<p>Each image stack has its own image count, seemingly a few hunderd TIFF images each (e.g., 270, 500, 309, etc.). Each stack’s images are all the same size but the sizes differ between stacks (e.g. 33MB images, 58MB images, etc.). Seemingly, on the order of 30 to 50 MB per image. </p>\n<p>One TEST<em>DATA</em>SET sample neuron’s data is a folder, named <code class=\"language-text\">665856925</code>:</p>\n<ul>\n<li>Full of about 280 TIFF images</li>\n<li>All files named like:<code class=\"language-text\">reconstruction_0_0539044525_639962984-0007.tif</code> </li>\n<li>The only thing that changes is the last four characters in the filename root, after the hyphen.</li>\n<li>Each file is about 33 MB in size</li>\n<li>One neuron’s data is on the order of 10 gigabyte</li>\n</ul>\n<h2>Colab can handle one neuron’s data at a time</h2>\n<p>Consider one large neuron, Name/ID of <code class=\"language-text\">647225829</code>. This one has 460 images, each 57.7MB. So, an average neuron’s data can be as big as, say, 25 gigabytes. They range from ~6GB to ~60GB (specimen 687746742 is 59.9GB)</p>\n<p>Fortuneately, Google’s Colab has that much file system. They give out 50GB file systems by default. And if you ask for a GPU they actually give you 350GB. </p>\n<p>350GB is enough file system to process the largest specimen in the dataset. Additionally, the U-Net implementation can use the T4 GPU. </p>\n<div class=\"gatsby-highlight\" data-language=\"text\"><pre class=\"language-text\"><code class=\"language-text\"># Get some stats on the file system:\n!!df -h .</code></pre></div>\n<div class=\"gatsby-highlight\" data-language=\"text\"><pre class=\"language-text\"><code class=\"language-text\">[&#39;Filesystem      Size  Used Avail Use% Mounted on&#39;,\n &#39;overlay          49G   25G   22G  54% /&#39;]</code></pre></div>\n<p>The default file system on Colab is 50G, but a 360G file system can be requested, simply by configuring the runtime to have a GPU (yup).</p>\n<p>So, on the default (25G) file system, half the file system is already used by the OS and other pre-installed software. A big neuron’s data would consume the remaining 25G. So <strong>probably a good idea to request a GPU</strong> which will also come with ~360G file system.</p>\n<h2>Overview of the dataset</h2>\n<p><strong>The goal here</strong> is to have a bit of utility code that completely maps the dataset’s file system, programmatically walking the file system. </p>\n<p>All this tedius code makes two things:</p>\n<ol>\n<li>training<em>neurons: dictionary (105 neurons) keyed by neuron</em>id </li>\n<li>testing<em>neurons: dictionary (10 neurons) keyed by neuron</em>id</li>\n</ol>\n<p>All 115 neurons and all their files (names and sizes) programmatically indexed into a convenient data structure with which to build out manifest files for, say, ShuTu or some U-Net reconstructor to process. I.e. this will make it easier for folks to massage the data into whatever tool they decide to run with.</p>\n<p>The data is stored on Wasabi Cloud Storage, which mimics the AWS S3 APIs, so AWS’s Python client, boto3, can be used to access the data. boto3 comes preinstalled on Colab. Here’s Wasabi’s how-to doc, <a href=\"https://wasabi-support.zendesk.com/hc/en-us/articles/115002579891-How-do-I-use-the-AWS-SDK-for-Python-boto3-with-Wasabi-\">How do I use the AWS SDK for Python (boto3) with Wasabi?\n</a></p>\n<h3>Set up shop</h3>\n<div class=\"gatsby-highlight\" data-language=\"python\"><pre class=\"language-python\"><code class=\"language-python\"><span class=\"token keyword\">import</span> boto3\n<span class=\"token keyword\">import</span> json\n<span class=\"token keyword\">import</span> os\n<span class=\"token keyword\">import</span> numpy <span class=\"token keyword\">as</span> np\n<span class=\"token keyword\">import</span> seaborn <span class=\"token keyword\">as</span> sns\n<span class=\"token keyword\">import</span> time\n<span class=\"token keyword\">from</span> IPython<span class=\"token punctuation\">.</span>display <span class=\"token keyword\">import</span> HTML<span class=\"token punctuation\">,</span> display\n\nsns<span class=\"token punctuation\">.</span><span class=\"token builtin\">set</span><span class=\"token punctuation\">(</span>color_codes<span class=\"token operator\">=</span><span class=\"token boolean\">True</span><span class=\"token punctuation\">)</span></code></pre></div>\n<div class=\"gatsby-highlight\" data-language=\"python\"><pre class=\"language-python\"><code class=\"language-python\"><span class=\"token comment\"># A Colab progress bar</span>\n\n<span class=\"token keyword\">def</span> <span class=\"token function\">progress</span><span class=\"token punctuation\">(</span>value<span class=\"token punctuation\">,</span> <span class=\"token builtin\">max</span><span class=\"token operator\">=</span><span class=\"token number\">100</span><span class=\"token punctuation\">)</span><span class=\"token punctuation\">:</span>\n    <span class=\"token keyword\">return</span> HTML<span class=\"token punctuation\">(</span><span class=\"token triple-quoted-string string\">\"\"\"\n        &lt;progress\n            value='{value}'\n            max='{max}',\n            style='width: 100%'\n        >\n            {value}\n        &lt;/progress>\n    \"\"\"</span><span class=\"token punctuation\">.</span><span class=\"token builtin\">format</span><span class=\"token punctuation\">(</span>value<span class=\"token operator\">=</span>value<span class=\"token punctuation\">,</span> <span class=\"token builtin\">max</span><span class=\"token operator\">=</span><span class=\"token builtin\">max</span><span class=\"token punctuation\">)</span><span class=\"token punctuation\">)</span></code></pre></div>\n<h3>Map the 105 training data neurons</h3>\n<p>This only pulls down the keys and metadata, not the actual images nor SWC files.</p>\n<div class=\"gatsby-highlight\" data-language=\"python\"><pre class=\"language-python\"><code class=\"language-python\"><span class=\"token comment\"># Tweaked out code via https://stackoverflow.com/a/49361727 and https://stackoverflow.com/a/14822210</span>\n<span class=\"token comment\"># TODO: test this. 2.5 vs. 2.7 TB was seen?</span>\n<span class=\"token keyword\">def</span> <span class=\"token function\">format_bytes</span><span class=\"token punctuation\">(</span>size<span class=\"token punctuation\">)</span><span class=\"token punctuation\">:</span>\n    <span class=\"token comment\"># 2**10 = 1024</span>\n    power <span class=\"token operator\">=</span> <span class=\"token number\">2</span><span class=\"token operator\">**</span><span class=\"token number\">10</span>\n    n <span class=\"token operator\">=</span> <span class=\"token number\">0</span>\n    power_labels <span class=\"token operator\">=</span> <span class=\"token punctuation\">{</span><span class=\"token number\">0</span> <span class=\"token punctuation\">:</span> <span class=\"token string\">''</span><span class=\"token punctuation\">,</span> <span class=\"token number\">1</span><span class=\"token punctuation\">:</span> <span class=\"token string\">'K'</span><span class=\"token punctuation\">,</span> <span class=\"token number\">2</span><span class=\"token punctuation\">:</span> <span class=\"token string\">'M'</span><span class=\"token punctuation\">,</span> <span class=\"token number\">3</span><span class=\"token punctuation\">:</span> <span class=\"token string\">'G'</span><span class=\"token punctuation\">,</span> <span class=\"token number\">4</span><span class=\"token punctuation\">:</span> <span class=\"token string\">'T'</span><span class=\"token punctuation\">}</span>\n    <span class=\"token keyword\">while</span> size <span class=\"token operator\">></span> power<span class=\"token punctuation\">:</span>\n        size <span class=\"token operator\">/=</span> power\n        n <span class=\"token operator\">+=</span> <span class=\"token number\">1</span>\n    <span class=\"token keyword\">return</span> size<span class=\"token punctuation\">,</span> power_labels<span class=\"token punctuation\">[</span>n<span class=\"token punctuation\">]</span><span class=\"token operator\">+</span><span class=\"token string\">'B'</span>\n    \n<span class=\"token keyword\">def</span> <span class=\"token function\">sumObjectsForPrefix</span><span class=\"token punctuation\">(</span>a_prefix<span class=\"token punctuation\">)</span><span class=\"token punctuation\">:</span>\n  <span class=\"token string\">\"sums gigabytes of file system occupied by all objects is a directory)\"</span>\n  tots <span class=\"token operator\">=</span> <span class=\"token number\">0</span>\n  tots <span class=\"token operator\">=</span> <span class=\"token builtin\">sum</span><span class=\"token punctuation\">(</span><span class=\"token number\">1</span> <span class=\"token keyword\">for</span> _ <span class=\"token keyword\">in</span> bucket<span class=\"token punctuation\">.</span>objects<span class=\"token punctuation\">.</span><span class=\"token builtin\">filter</span><span class=\"token punctuation\">(</span>Prefix <span class=\"token operator\">=</span> a_prefix<span class=\"token punctuation\">)</span><span class=\"token punctuation\">)</span> \n  <span class=\"token keyword\">return</span> tots\n\ns3 <span class=\"token operator\">=</span> boto3<span class=\"token punctuation\">.</span>resource<span class=\"token punctuation\">(</span><span class=\"token string\">'s3'</span><span class=\"token punctuation\">,</span>\n     endpoint_url <span class=\"token operator\">=</span> <span class=\"token string\">'https://s3.us-west-1.wasabisys.com'</span><span class=\"token punctuation\">,</span>\n     aws_access_key_id <span class=\"token operator\">=</span> <span class=\"token string\">'2G7POM6IZKJ3KLHSC4JB'</span><span class=\"token punctuation\">,</span>\n     aws_secret_access_key <span class=\"token operator\">=</span> <span class=\"token string\">\"0oHD5BXPim7fR1n7zDXpz4YoB7CHAHAvFgzpuJnt\"</span><span class=\"token punctuation\">)</span>  \nbucket <span class=\"token operator\">=</span> s3<span class=\"token punctuation\">.</span>Bucket<span class=\"token punctuation\">(</span><span class=\"token string\">'brightfield-auto-reconstruction-competition'</span><span class=\"token punctuation\">)</span>\n\nresult <span class=\"token operator\">=</span> bucket<span class=\"token punctuation\">.</span>meta<span class=\"token punctuation\">.</span>client<span class=\"token punctuation\">.</span>list_objects<span class=\"token punctuation\">(</span>Bucket<span class=\"token operator\">=</span>bucket<span class=\"token punctuation\">.</span>name<span class=\"token punctuation\">,</span>\n                                         Delimiter<span class=\"token operator\">=</span><span class=\"token string\">'/'</span><span class=\"token punctuation\">)</span>\n<span class=\"token keyword\">print</span><span class=\"token punctuation\">(</span> <span class=\"token string\">\"Total root subfolders = \"</span> <span class=\"token operator\">+</span> <span class=\"token builtin\">str</span><span class=\"token punctuation\">(</span><span class=\"token builtin\">sum</span><span class=\"token punctuation\">(</span><span class=\"token number\">1</span> <span class=\"token keyword\">for</span> _ <span class=\"token keyword\">in</span> result<span class=\"token punctuation\">.</span>get<span class=\"token punctuation\">(</span><span class=\"token string\">'CommonPrefixes'</span><span class=\"token punctuation\">)</span> <span class=\"token punctuation\">)</span><span class=\"token punctuation\">)</span> <span class=\"token operator\">+</span> <span class=\"token string\">\". Mapping training image stacks, one at a time...\"</span><span class=\"token punctuation\">)</span>\n\n<span class=\"token comment\"># Walk the dataset file system. First the 105 training TIFF stacks, with SWCs                    </span>\n\n<span class=\"token comment\"># TODO: kill this off once find bug. [What bug, damnit]</span>\ntotal_bytes_in_training_specimens <span class=\"token operator\">=</span> <span class=\"token number\">0</span>\ntotal_files_in_training_cells <span class=\"token operator\">=</span> <span class=\"token number\">0</span>\n\n<span class=\"token comment\"># Set up a progress indicator for this slow task:</span>\nprogressIndicator <span class=\"token operator\">=</span> display<span class=\"token punctuation\">(</span>progress<span class=\"token punctuation\">(</span><span class=\"token number\">0</span><span class=\"token punctuation\">,</span> <span class=\"token number\">100</span><span class=\"token punctuation\">)</span><span class=\"token punctuation\">,</span> display_id<span class=\"token operator\">=</span><span class=\"token boolean\">True</span><span class=\"token punctuation\">)</span>\nprogressIndicator_count <span class=\"token operator\">=</span> <span class=\"token number\">0</span>\nprogressIndicator_end <span class=\"token operator\">=</span> <span class=\"token number\">105</span>\n\ntraining_neurons <span class=\"token operator\">=</span> <span class=\"token punctuation\">{</span><span class=\"token punctuation\">}</span>\n<span class=\"token keyword\">for</span> o <span class=\"token keyword\">in</span> result<span class=\"token punctuation\">.</span>get<span class=\"token punctuation\">(</span><span class=\"token string\">'CommonPrefixes'</span><span class=\"token punctuation\">)</span><span class=\"token punctuation\">:</span>\n  progressIndicator_count <span class=\"token operator\">+=</span> <span class=\"token number\">1</span>\n  progressIndicator<span class=\"token punctuation\">.</span>update<span class=\"token punctuation\">(</span>progress<span class=\"token punctuation\">(</span>progressIndicator_count<span class=\"token punctuation\">,</span> progressIndicator_end<span class=\"token punctuation\">)</span><span class=\"token punctuation\">)</span>\n  a_prefix <span class=\"token operator\">=</span> o<span class=\"token punctuation\">.</span>get<span class=\"token punctuation\">(</span><span class=\"token string\">'Prefix'</span><span class=\"token punctuation\">)</span>\n  <span class=\"token comment\"># 106 lines of random numbers: </span>\n  <span class=\"token comment\">#print(a_prefix)</span>\n  \n  <span class=\"token comment\"># Enumerate all files</span>\n  <span class=\"token comment\"># print(\"----------------\")</span>\n  imagestack_bytes <span class=\"token operator\">=</span> <span class=\"token number\">0</span>\n  imagestack <span class=\"token operator\">=</span> <span class=\"token punctuation\">[</span><span class=\"token punctuation\">]</span>\n  swc_key <span class=\"token operator\">=</span> <span class=\"token boolean\">None</span>\n  <span class=\"token keyword\">for</span> s3_object <span class=\"token keyword\">in</span> bucket<span class=\"token punctuation\">.</span>objects<span class=\"token punctuation\">.</span><span class=\"token builtin\">filter</span><span class=\"token punctuation\">(</span>Prefix <span class=\"token operator\">=</span> a_prefix<span class=\"token punctuation\">)</span><span class=\"token punctuation\">:</span>\n    <span class=\"token comment\"># print(s3_object.key + \"= \" + str(s3_object.size))</span>\n    total_files_in_training_cells <span class=\"token operator\">+=</span> <span class=\"token number\">1</span>\n    <span class=\"token keyword\">if</span> <span class=\"token keyword\">not</span> s3_object<span class=\"token punctuation\">.</span>key<span class=\"token punctuation\">.</span>endswith<span class=\"token punctuation\">(</span><span class=\"token string\">\".swc\"</span><span class=\"token punctuation\">)</span><span class=\"token punctuation\">:</span>\n      <span class=\"token keyword\">if</span> s3_object<span class=\"token punctuation\">.</span>key <span class=\"token operator\">!=</span> a_prefix<span class=\"token punctuation\">:</span>\n        <span class=\"token comment\"># if == it's the directory itself, not a file in it so ignore</span>\n        imagestack<span class=\"token punctuation\">.</span>append<span class=\"token punctuation\">(</span>s3_object<span class=\"token punctuation\">.</span>key<span class=\"token punctuation\">)</span>\n        imagestack_bytes <span class=\"token operator\">+=</span> s3_object<span class=\"token punctuation\">.</span>size\n        total_bytes_in_training_specimens <span class=\"token operator\">+=</span> s3_object<span class=\"token punctuation\">.</span>size\n    <span class=\"token keyword\">else</span><span class=\"token punctuation\">:</span>\n      swc_key <span class=\"token operator\">=</span> s3_object<span class=\"token punctuation\">.</span>key\n  \n  <span class=\"token keyword\">if</span> a_prefix <span class=\"token operator\">!=</span> <span class=\"token string\">\"TEST_DATA_SET/\"</span><span class=\"token punctuation\">:</span>\n    specimen_id <span class=\"token operator\">=</span> a_prefix<span class=\"token punctuation\">[</span><span class=\"token punctuation\">:</span><span class=\"token operator\">-</span><span class=\"token number\">1</span><span class=\"token punctuation\">]</span> <span class=\"token comment\"># get rid of trailing /</span>\n    training_neurons<span class=\"token punctuation\">[</span>specimen_id<span class=\"token punctuation\">]</span> <span class=\"token operator\">=</span> <span class=\"token punctuation\">{</span><span class=\"token string\">\"prefix\"</span><span class=\"token punctuation\">:</span> a_prefix<span class=\"token punctuation\">,</span> <span class=\"token string\">\"swc\"</span><span class=\"token punctuation\">:</span> swc_key<span class=\"token punctuation\">,</span> <span class=\"token string\">\"imagestack\"</span><span class=\"token punctuation\">:</span> imagestack<span class=\"token punctuation\">,</span> <span class=\"token string\">\"size\"</span><span class=\"token punctuation\">:</span> imagestack_bytes<span class=\"token punctuation\">}</span>\n        \n<span class=\"token keyword\">print</span><span class=\"token punctuation\">(</span><span class=\"token string\">\"Training neurons mapped: \"</span> <span class=\"token operator\">+</span> <span class=\"token builtin\">str</span><span class=\"token punctuation\">(</span><span class=\"token builtin\">len</span><span class=\"token punctuation\">(</span>training_neurons<span class=\"token punctuation\">)</span><span class=\"token punctuation\">)</span><span class=\"token punctuation\">)</span>    \ntraining_files_size<span class=\"token punctuation\">,</span> training_files_units <span class=\"token operator\">=</span> format_bytes<span class=\"token punctuation\">(</span>total_bytes_in_training_specimens<span class=\"token punctuation\">)</span>\n<span class=\"token keyword\">print</span><span class=\"token punctuation\">(</span><span class=\"token string\">\"Summed file size of all training cells: %s %s (%d bytes, %d files)\"</span> <span class=\"token operator\">%</span>  <span class=\"token punctuation\">(</span><span class=\"token string\">'{:4.1f}'</span><span class=\"token punctuation\">.</span><span class=\"token builtin\">format</span><span class=\"token punctuation\">(</span>training_files_size<span class=\"token punctuation\">)</span><span class=\"token punctuation\">,</span> training_files_units<span class=\"token punctuation\">,</span> total_bytes_in_training_specimens<span class=\"token punctuation\">,</span> total_files_in_training_cells<span class=\"token punctuation\">)</span><span class=\"token punctuation\">)</span></code></pre></div>\n<div class=\"gatsby-highlight\" data-language=\"text\"><pre class=\"language-text\"><code class=\"language-text\">Total root subfolders = 106. Mapping training image stacks, one at a time...\n...\nTraining neurons mapped: 105\nSummed file size of all training cells:  2.5 TB (2713760166906 bytes, 53926 files)</code></pre></div>\n<p>106 folders for 105 training neurons and the last folder is <code class=\"language-text\">TEST_DATA_SET</code> which contains 10 neuron image stacks in subfolders (without SWC answers).</p>\n<p>Whelp, time and space are limited on Colab so let’s figure out which neurons are the smallest ergo the fasted to process (hopefully).</p>\n<h3>List training neurons by file size</h3>\n<div class=\"gatsby-highlight\" data-language=\"python\"><pre class=\"language-python\"><code class=\"language-python\"><span class=\"token comment\"># bitwize shift 30 converts bytes to gigabytes</span>\ntraining_cell_sizes <span class=\"token operator\">=</span> np<span class=\"token punctuation\">.</span>array<span class=\"token punctuation\">(</span><span class=\"token punctuation\">[</span>cell<span class=\"token punctuation\">[</span><span class=\"token string\">\"size\"</span><span class=\"token punctuation\">]</span><span class=\"token operator\">>></span><span class=\"token number\">30</span> <span class=\"token keyword\">for</span> cell <span class=\"token keyword\">in</span> training_neurons<span class=\"token punctuation\">.</span>values<span class=\"token punctuation\">(</span><span class=\"token punctuation\">)</span><span class=\"token punctuation\">]</span><span class=\"token punctuation\">)</span>\nsizes_histogram <span class=\"token operator\">=</span> sns<span class=\"token punctuation\">.</span>distplot<span class=\"token punctuation\">(</span>training_cell_sizes<span class=\"token punctuation\">,</span> bins<span class=\"token operator\">=</span><span class=\"token number\">20</span><span class=\"token punctuation\">,</span> kde<span class=\"token operator\">=</span><span class=\"token boolean\">False</span><span class=\"token punctuation\">,</span> rug<span class=\"token operator\">=</span><span class=\"token boolean\">True</span><span class=\"token punctuation\">)</span><span class=\"token punctuation\">.</span>set_title<span class=\"token punctuation\">(</span><span class=\"token string\">\"Training cells image stacks (gigabytes)\"</span><span class=\"token punctuation\">)</span></code></pre></div>\n<p><span\n      class=\"gatsby-resp-image-wrapper\"\n      style=\"position: relative; display: block; margin-left: auto; margin-right: auto;  max-width: 383px;\"\n    >\n      <a\n    class=\"gatsby-resp-image-link\"\n    href=\"/static/b2acaceeb8dfaddf290a1c8449c28ffa/b4d9b/index_15_0.png\"\n    style=\"display: block\"\n    target=\"_blank\"\n    rel=\"noopener\"\n  >\n    <span\n    class=\"gatsby-resp-image-background-image\"\n    style=\"padding-bottom: 69.71279373368147%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAOCAYAAAAvxDzwAAAACXBIWXMAAAsSAAALEgHS3X78AAACUUlEQVQ4y32UiW7aQBCGef/HqFS16q02SqtQpWkISYQSCE25w+nbGJ/rAxP6d2ZtEI3aWlpmPLDf/DOzSwX755f8XK/XUBQFhqHDsizpW5YJVVWhaZq0HJ/P55hOp9B1HXm+3jMqaZoiSRLEcQbfTxAECZZLH7OFDdN0Ydu+fDcMh0AevRdL05b0/Ur6nhfLvczYAzloWQFtDmFaIU4bM2hGAMeJKBaRFdLu/MNl0e95r+/HqIRhSKoCqYyDritwdafg49mQwAXQpiScyLaLxdDChmWSIs6MSpZlSNNESpZBynJ+M8en7yMMJ0u0ugZWq2gPOwQ+9SVQCAFeYZhiprjoj5eotxY4ImBnZKHeXEjVh9D/ArnkMIogogyDsS1hXDIr7I5sXLdVuCsh1Zplj5+W/AeQS2aFESlkIAN2PRyQWra9BxtnNCRF80rIYfmFLYZCwO12i5imLKIU45mDS4JdtgrgaOrg1UlP9pFjqu7BKSf9T4V8kCMqmem3P3W8Px3g7dc+jmsPEvi62kO7Z+CCerlTuJv6X4F5vkFKCm06Hh++DfCuBLLC6tUEb6p9tPumVGiageynR0Nq/KCbo/s0LLGHl8BcAi3KwOW9/NLF8+MOXrD93MGzo3u0OjpOLie4pnbwcDpDS1YwV1ypTKNWqFpxYyoZlSy4ZBrKBamoNeeokWX//Jb8poIJbWzcq2j1TExVF/W7BVqk2vUEFIIpdKPGixVWpFwOhVXys33MkcSi9DeIRSj9R/KjKJD+ZpPTiSh8/lMIAr+IE8PzPPwGnzIOO+2GTKYAAAAASUVORK5CYII='); background-size: cover; display: block;\"\n  ></span>\n  <img\n        class=\"gatsby-resp-image-image\"\n        alt=\"png\"\n        title=\"png\"\n        src=\"/static/b2acaceeb8dfaddf290a1c8449c28ffa/b4d9b/index_15_0.png\"\n        srcset=\"/static/b2acaceeb8dfaddf290a1c8449c28ffa/1abb1/index_15_0.png 250w,\n/static/b2acaceeb8dfaddf290a1c8449c28ffa/b4d9b/index_15_0.png 383w\"\n        sizes=\"(max-width: 383px) 100vw, 383px\"\n        loading=\"lazy\"\n      />\n  </a>\n    </span></p>\n<div class=\"gatsby-highlight\" data-language=\"python\"><pre class=\"language-python\"><code class=\"language-python\"><span class=\"token comment\"># List cell sorted by fileset size (z-stack and SWC), plus averages and total</span>\n\n<span class=\"token keyword\">def</span> <span class=\"token function\">sizer</span><span class=\"token punctuation\">(</span>x<span class=\"token punctuation\">)</span><span class=\"token punctuation\">:</span> \n  <span class=\"token keyword\">return</span> training_neurons<span class=\"token punctuation\">[</span>x<span class=\"token punctuation\">]</span><span class=\"token punctuation\">[</span><span class=\"token string\">\"size\"</span><span class=\"token punctuation\">]</span>\n\nsize_sorted <span class=\"token operator\">=</span> <span class=\"token builtin\">sorted</span><span class=\"token punctuation\">(</span>training_neurons<span class=\"token punctuation\">,</span> key <span class=\"token operator\">=</span> sizer<span class=\"token punctuation\">)</span> \ntotal_bytes_in_training_dataset <span class=\"token operator\">=</span> <span class=\"token number\">0</span>    \ntotal_training_specimens <span class=\"token operator\">=</span> <span class=\"token number\">0</span>  \n  \n<span class=\"token keyword\">for</span> a_neuron_name <span class=\"token keyword\">in</span> size_sorted<span class=\"token punctuation\">:</span>\n  total_training_specimens <span class=\"token operator\">+=</span> <span class=\"token number\">1</span>\n  a_neuron <span class=\"token operator\">=</span> training_neurons<span class=\"token punctuation\">[</span>a_neuron_name<span class=\"token punctuation\">]</span>\n  fileSize<span class=\"token punctuation\">,</span> fileUnits <span class=\"token operator\">=</span> format_bytes<span class=\"token punctuation\">(</span>a_neuron<span class=\"token punctuation\">[</span><span class=\"token string\">\"size\"</span><span class=\"token punctuation\">]</span><span class=\"token punctuation\">)</span>\n  total_bytes_in_training_dataset <span class=\"token operator\">+=</span> a_neuron<span class=\"token punctuation\">[</span><span class=\"token string\">\"size\"</span><span class=\"token punctuation\">]</span>\n  <span class=\"token keyword\">print</span><span class=\"token punctuation\">(</span>a_neuron_name <span class=\"token operator\">+</span> <span class=\"token string\">\": \"</span> <span class=\"token operator\">+</span> <span class=\"token builtin\">str</span><span class=\"token punctuation\">(</span><span class=\"token builtin\">len</span><span class=\"token punctuation\">(</span>a_neuron<span class=\"token punctuation\">[</span><span class=\"token string\">\"imagestack\"</span><span class=\"token punctuation\">]</span><span class=\"token punctuation\">)</span><span class=\"token punctuation\">)</span> <span class=\"token operator\">+</span> <span class=\"token string\">\" files = \"</span> <span class=\"token operator\">+</span> <span class=\"token string\">'{:4.1f}'</span><span class=\"token punctuation\">.</span><span class=\"token builtin\">format</span><span class=\"token punctuation\">(</span>fileSize<span class=\"token punctuation\">)</span> <span class=\"token operator\">+</span> <span class=\"token string\">\" \"</span> <span class=\"token operator\">+</span> fileUnits <span class=\"token punctuation\">)</span>\n\naverage_specimen_size <span class=\"token operator\">=</span> total_bytes_in_training_dataset <span class=\"token operator\">/</span> total_training_specimens\naverage_size<span class=\"token punctuation\">,</span> averages_unit <span class=\"token operator\">=</span> format_bytes<span class=\"token punctuation\">(</span>average_specimen_size<span class=\"token punctuation\">)</span>\n\ntotal_file_size<span class=\"token punctuation\">,</span> total_file_unit <span class=\"token operator\">=</span> format_bytes<span class=\"token punctuation\">(</span>total_bytes_in_training_dataset<span class=\"token punctuation\">)</span>\n<span class=\"token keyword\">print</span><span class=\"token punctuation\">(</span><span class=\"token string\">\"\\nNumber of cells in training dataset = %d\"</span> <span class=\"token operator\">%</span> total_training_specimens<span class=\"token punctuation\">)</span>\n<span class=\"token keyword\">print</span><span class=\"token punctuation\">(</span><span class=\"token string\">\"Average cell data size = \"</span> <span class=\"token operator\">+</span> <span class=\"token string\">'{:4.1f}'</span><span class=\"token punctuation\">.</span><span class=\"token builtin\">format</span><span class=\"token punctuation\">(</span>average_size<span class=\"token punctuation\">)</span> <span class=\"token operator\">+</span> <span class=\"token string\">\" \"</span> <span class=\"token operator\">+</span> averages_unit <span class=\"token operator\">+</span> <span class=\"token string\">\" (\"</span> <span class=\"token operator\">+</span> <span class=\"token builtin\">str</span><span class=\"token punctuation\">(</span><span class=\"token builtin\">int</span><span class=\"token punctuation\">(</span>average_specimen_size<span class=\"token punctuation\">)</span><span class=\"token punctuation\">)</span> <span class=\"token operator\">+</span> <span class=\"token string\">\" bytes)\"</span><span class=\"token punctuation\">)</span>\n<span class=\"token keyword\">print</span><span class=\"token punctuation\">(</span><span class=\"token string\">\"Total size of training dataset = \"</span> <span class=\"token operator\">+</span> <span class=\"token string\">'{:4.1f}'</span><span class=\"token punctuation\">.</span><span class=\"token builtin\">format</span><span class=\"token punctuation\">(</span>total_file_size<span class=\"token punctuation\">)</span> <span class=\"token operator\">+</span> <span class=\"token string\">\" \"</span> <span class=\"token operator\">+</span> total_file_unit <span class=\"token operator\">+</span> <span class=\"token string\">\" (\"</span> <span class=\"token operator\">+</span> <span class=\"token builtin\">str</span><span class=\"token punctuation\">(</span>total_bytes_in_training_dataset<span class=\"token punctuation\">)</span> <span class=\"token operator\">+</span> <span class=\"token string\">\" bytes)\"</span><span class=\"token punctuation\">)</span></code></pre></div>\n<div class=\"gatsby-highlight\" data-language=\"text\"><pre class=\"language-text\"><code class=\"language-text\">651806289: 291 files =  6.0 GB\n647289876: 228 files =  7.0 GB\n651748297: 336 files =  7.0 GB\n...\n697851947: 850 files = 45.7 GB\n699189400: 650 files = 53.8 GB\n687746742: 608 files = 59.9 GB\n\nNumber of cells in training dataset = 105\nAverage cell data size = 21.6 GB (23172964855 bytes)\nTotal size of training dataset =  2.2 TB (2433161309799 bytes)</code></pre></div>\n<p>In summary, there are 105 training neurons. The specimens’ size range from 6.0 GB to 59.9 GB. Seven specimens are smaller than 10 GB.</p>\n<h3>Map the 10 testing neuron</h3>\n<p>The final part of the challenge data set to be mapped is the sub-root directory, <code class=\"language-text\">TEST_DATA_SET</code>, which has 10 neurons laid out like with the training data, except the SWC files are missing i.e. no reconstruction answers given (because, that is what the challenger is supposed to demonstrate: the capability to generate quality SWC files).</p>\n<div class=\"gatsby-highlight\" data-language=\"python\"><pre class=\"language-python\"><code class=\"language-python\">client <span class=\"token operator\">=</span> boto3<span class=\"token punctuation\">.</span>client<span class=\"token punctuation\">(</span><span class=\"token string\">'s3'</span><span class=\"token punctuation\">,</span>\n     endpoint_url <span class=\"token operator\">=</span> <span class=\"token string\">'https://s3.us-west-1.wasabisys.com'</span><span class=\"token punctuation\">,</span>\n     aws_access_key_id <span class=\"token operator\">=</span> <span class=\"token string\">''</span><span class=\"token punctuation\">,</span>\n     aws_secret_access_key <span class=\"token operator\">=</span> <span class=\"token string\">\"\"</span><span class=\"token punctuation\">)</span>\npaginator <span class=\"token operator\">=</span> client<span class=\"token punctuation\">.</span>get_paginator<span class=\"token punctuation\">(</span><span class=\"token string\">'list_objects'</span><span class=\"token punctuation\">)</span>\nresult <span class=\"token operator\">=</span> paginator<span class=\"token punctuation\">.</span>paginate<span class=\"token punctuation\">(</span>\n    Bucket<span class=\"token operator\">=</span><span class=\"token string\">'brightfield-auto-reconstruction-competition'</span><span class=\"token punctuation\">,</span> \n    Prefix<span class=\"token operator\">=</span><span class=\"token string\">\"TEST_DATA_SET/\"</span><span class=\"token punctuation\">,</span> \n    Delimiter<span class=\"token operator\">=</span><span class=\"token string\">'/'</span><span class=\"token punctuation\">)</span>\n    <span class=\"token comment\"># See https://stackoverflow.com/a/36992023</span>\n    <span class=\"token comment\"># A response can contain CommonPrefixes only if you specify a delimiter. When you do, CommonPrefixes contains all (if there are any) keys between Prefix and the next occurrence of the string specified by delimiter. In effect, CommonPrefixes lists keys that act like subdirectories in the directory specified by Prefix.</span>\n\n<span class=\"token comment\">#for prefix in result.search('CommonPrefixes'):</span>\n<span class=\"token comment\">#    print(prefix.get('Prefix'))</span>\n    \ntesting_neurons <span class=\"token operator\">=</span> <span class=\"token punctuation\">{</span><span class=\"token punctuation\">}</span>\n\n<span class=\"token comment\"># Set up a progress indicator for this slow but not too slow task:</span>\nprogressIndicator <span class=\"token operator\">=</span> display<span class=\"token punctuation\">(</span>progress<span class=\"token punctuation\">(</span><span class=\"token number\">0</span><span class=\"token punctuation\">,</span> <span class=\"token number\">10</span><span class=\"token punctuation\">)</span><span class=\"token punctuation\">,</span> display_id<span class=\"token operator\">=</span><span class=\"token boolean\">True</span><span class=\"token punctuation\">)</span>\nprogressIndicator_count <span class=\"token operator\">=</span> <span class=\"token number\">0</span>\nprogressIndicator_end <span class=\"token operator\">=</span> <span class=\"token number\">10</span>\n\n<span class=\"token keyword\">for</span> o <span class=\"token keyword\">in</span> result<span class=\"token punctuation\">.</span>search<span class=\"token punctuation\">(</span><span class=\"token string\">'CommonPrefixes'</span><span class=\"token punctuation\">)</span><span class=\"token punctuation\">:</span>\n  progressIndicator_count <span class=\"token operator\">+=</span> <span class=\"token number\">1</span>\n  progressIndicator<span class=\"token punctuation\">.</span>update<span class=\"token punctuation\">(</span>progress<span class=\"token punctuation\">(</span>progressIndicator_count<span class=\"token punctuation\">,</span> progressIndicator_end<span class=\"token punctuation\">)</span><span class=\"token punctuation\">)</span>\n  a_prefix <span class=\"token operator\">=</span> o<span class=\"token punctuation\">.</span>get<span class=\"token punctuation\">(</span><span class=\"token string\">\"Prefix\"</span><span class=\"token punctuation\">)</span>\n  <span class=\"token keyword\">print</span><span class=\"token punctuation\">(</span>a_prefix<span class=\"token punctuation\">)</span>\n  \n  <span class=\"token comment\"># Enumerate all files</span>\n  <span class=\"token comment\"># print(\"----------------\")</span>\n  imagestack_bytes <span class=\"token operator\">=</span> <span class=\"token number\">0</span>\n  imagestack <span class=\"token operator\">=</span> <span class=\"token punctuation\">[</span><span class=\"token punctuation\">]</span>\n  swc_key <span class=\"token operator\">=</span> <span class=\"token boolean\">None</span>\n  <span class=\"token keyword\">for</span> s3_object <span class=\"token keyword\">in</span> bucket<span class=\"token punctuation\">.</span>objects<span class=\"token punctuation\">.</span><span class=\"token builtin\">filter</span><span class=\"token punctuation\">(</span>Prefix <span class=\"token operator\">=</span> a_prefix<span class=\"token punctuation\">)</span><span class=\"token punctuation\">:</span>\n    <span class=\"token comment\"># print(s3_object.key + \"= \" + str(s3_object.size))</span>\n    <span class=\"token keyword\">if</span> <span class=\"token keyword\">not</span> s3_object<span class=\"token punctuation\">.</span>key<span class=\"token punctuation\">.</span>endswith<span class=\"token punctuation\">(</span><span class=\"token string\">\".swc\"</span><span class=\"token punctuation\">)</span><span class=\"token punctuation\">:</span>\n      <span class=\"token keyword\">if</span> s3_object<span class=\"token punctuation\">.</span>key <span class=\"token operator\">!=</span> a_prefix<span class=\"token punctuation\">:</span>\n        <span class=\"token comment\"># if == it's the directory itself, not a file in it so ignore</span>\n        imagestack<span class=\"token punctuation\">.</span>append<span class=\"token punctuation\">(</span>s3_object<span class=\"token punctuation\">.</span>key<span class=\"token punctuation\">)</span>\n        imagestack_bytes <span class=\"token operator\">+=</span> s3_object<span class=\"token punctuation\">.</span>size\n    <span class=\"token keyword\">else</span><span class=\"token punctuation\">:</span>\n      swc_key <span class=\"token operator\">=</span> s3_object<span class=\"token punctuation\">.</span>key\n  \n  <span class=\"token comment\"># Strip the \"TEST_DATA_SET/\" and trailing \"/\" from Prefix</span>\n  neuron_id <span class=\"token operator\">=</span> a_prefix<span class=\"token punctuation\">[</span><span class=\"token builtin\">len</span><span class=\"token punctuation\">(</span><span class=\"token string\">\"TEST_DATA_SET/\"</span><span class=\"token punctuation\">)</span><span class=\"token punctuation\">:</span><span class=\"token operator\">-</span><span class=\"token number\">1</span><span class=\"token punctuation\">]</span>\n  \n  testing_neurons<span class=\"token punctuation\">[</span>neuron_id<span class=\"token punctuation\">]</span> <span class=\"token operator\">=</span> <span class=\"token punctuation\">{</span><span class=\"token string\">\"prefix\"</span><span class=\"token punctuation\">:</span> a_prefix<span class=\"token punctuation\">,</span> <span class=\"token string\">\"swc\"</span><span class=\"token punctuation\">:</span> swc_key<span class=\"token punctuation\">,</span> <span class=\"token string\">\"imagestack\"</span><span class=\"token punctuation\">:</span> imagestack<span class=\"token punctuation\">,</span> <span class=\"token string\">\"size\"</span><span class=\"token punctuation\">:</span> imagestack_bytes<span class=\"token punctuation\">}</span>\n        \n<span class=\"token keyword\">print</span><span class=\"token punctuation\">(</span> <span class=\"token string\">\"# testing neurons mapped: \"</span> <span class=\"token operator\">+</span> <span class=\"token builtin\">str</span><span class=\"token punctuation\">(</span><span class=\"token builtin\">len</span><span class=\"token punctuation\">(</span>testing_neurons<span class=\"token punctuation\">)</span><span class=\"token punctuation\">)</span> <span class=\"token operator\">+</span> <span class=\"token string\">\"\\nSorted by size of image stack:\"</span><span class=\"token punctuation\">)</span>    \n    \n<span class=\"token keyword\">def</span> <span class=\"token function\">testing_sizer</span><span class=\"token punctuation\">(</span>x<span class=\"token punctuation\">)</span><span class=\"token punctuation\">:</span> \n  <span class=\"token keyword\">return</span> testing_neurons<span class=\"token punctuation\">[</span>x<span class=\"token punctuation\">]</span><span class=\"token punctuation\">[</span><span class=\"token string\">\"size\"</span><span class=\"token punctuation\">]</span>\n\nsize_sorted_testing_neurons <span class=\"token operator\">=</span> <span class=\"token builtin\">sorted</span><span class=\"token punctuation\">(</span>testing_neurons<span class=\"token punctuation\">,</span> key <span class=\"token operator\">=</span> testing_sizer<span class=\"token punctuation\">)</span> \ntotal_bytes_in_testing_dataset <span class=\"token operator\">=</span> <span class=\"token number\">0</span>\n\n<span class=\"token keyword\">for</span> a_neuron_name <span class=\"token keyword\">in</span> size_sorted_testing_neurons<span class=\"token punctuation\">:</span>\n  a_neuron <span class=\"token operator\">=</span> testing_neurons<span class=\"token punctuation\">[</span>a_neuron_name<span class=\"token punctuation\">]</span>\n  fileSize<span class=\"token punctuation\">,</span> fileUnits <span class=\"token operator\">=</span> format_bytes<span class=\"token punctuation\">(</span>a_neuron<span class=\"token punctuation\">[</span><span class=\"token string\">\"size\"</span><span class=\"token punctuation\">]</span><span class=\"token punctuation\">)</span>\n  total_bytes_in_testing_dataset <span class=\"token operator\">+=</span> a_neuron<span class=\"token punctuation\">[</span><span class=\"token string\">\"size\"</span><span class=\"token punctuation\">]</span>\n  <span class=\"token keyword\">print</span><span class=\"token punctuation\">(</span>a_neuron_name <span class=\"token operator\">+</span> <span class=\"token string\">\": \"</span> <span class=\"token operator\">+</span> <span class=\"token builtin\">str</span><span class=\"token punctuation\">(</span><span class=\"token builtin\">len</span><span class=\"token punctuation\">(</span>a_neuron<span class=\"token punctuation\">[</span><span class=\"token string\">\"imagestack\"</span><span class=\"token punctuation\">]</span><span class=\"token punctuation\">)</span><span class=\"token punctuation\">)</span> <span class=\"token operator\">+</span> <span class=\"token string\">\" files = \"</span> <span class=\"token operator\">+</span> <span class=\"token string\">'{:4.1f}'</span><span class=\"token punctuation\">.</span><span class=\"token builtin\">format</span><span class=\"token punctuation\">(</span>fileSize<span class=\"token punctuation\">)</span> <span class=\"token operator\">+</span> <span class=\"token string\">\" \"</span> <span class=\"token operator\">+</span> fileUnits <span class=\"token punctuation\">)</span>\n\nfileSize<span class=\"token punctuation\">,</span> fileUnits <span class=\"token operator\">=</span> format_bytes<span class=\"token punctuation\">(</span>total_bytes_in_testing_dataset<span class=\"token punctuation\">)</span>\n<span class=\"token keyword\">print</span><span class=\"token punctuation\">(</span><span class=\"token string\">\"\\nTotal size of testing dataset = \"</span> <span class=\"token operator\">+</span> <span class=\"token string\">'{:4.1f}'</span><span class=\"token punctuation\">.</span><span class=\"token builtin\">format</span><span class=\"token punctuation\">(</span>fileSize<span class=\"token punctuation\">)</span> <span class=\"token operator\">+</span> <span class=\"token string\">\" \"</span> <span class=\"token operator\">+</span> fileUnits <span class=\"token punctuation\">)</span>  </code></pre></div>\n<div class=\"gatsby-highlight\" data-language=\"text\"><pre class=\"language-text\"><code class=\"language-text\">TEST_DATA_SET/665856925/\nTEST_DATA_SET/687730329/\nTEST_DATA_SET/691311995/\nTEST_DATA_SET/715953708/\nTEST_DATA_SET/741428906/\nTEST_DATA_SET/751017870/\nTEST_DATA_SET/761936495/\nTEST_DATA_SET/827413048/\nTEST_DATA_SET/850675694/\nTEST_DATA_SET/878858275/\n# testing neurons mapped: 10\nSorted by size of image stack:\n665856925: 281 files =  8.6 GB\n715953708: 340 files = 10.4 GB\n751017870: 465 files = 18.9 GB\n687730329: 497 files = 20.3 GB\n850675694: 438 files = 23.5 GB\n827413048: 424 files = 28.3 GB\n761936495: 529 files = 28.5 GB\n691311995: 441 files = 29.4 GB\n741428906: 591 files = 39.4 GB\n878858275: 541 files = 54.0 GB\n\nTotal size of testing dataset = 261.3 GB</code></pre></div>\n<div class=\"gatsby-highlight\" data-language=\"python\"><pre class=\"language-python\"><code class=\"language-python\"><span class=\"token comment\"># bitwize shift 30 converts bytes to gigabytes</span>\ntesting_cell_sizes <span class=\"token operator\">=</span> np<span class=\"token punctuation\">.</span>array<span class=\"token punctuation\">(</span><span class=\"token punctuation\">[</span>cell<span class=\"token punctuation\">[</span><span class=\"token string\">\"size\"</span><span class=\"token punctuation\">]</span><span class=\"token operator\">>></span><span class=\"token number\">30</span> <span class=\"token keyword\">for</span> cell <span class=\"token keyword\">in</span> testing_neurons<span class=\"token punctuation\">.</span>values<span class=\"token punctuation\">(</span><span class=\"token punctuation\">)</span><span class=\"token punctuation\">]</span><span class=\"token punctuation\">)</span>\nsizes_histogram <span class=\"token operator\">=</span> sns<span class=\"token punctuation\">.</span>distplot<span class=\"token punctuation\">(</span>testing_cell_sizes<span class=\"token punctuation\">,</span> bins<span class=\"token operator\">=</span><span class=\"token number\">20</span><span class=\"token punctuation\">,</span> kde<span class=\"token operator\">=</span><span class=\"token boolean\">False</span><span class=\"token punctuation\">,</span> rug<span class=\"token operator\">=</span><span class=\"token boolean\">True</span><span class=\"token punctuation\">)</span><span class=\"token punctuation\">.</span>set_title<span class=\"token punctuation\">(</span><span class=\"token string\">\"Test cells image stacks (gigabytes)\"</span><span class=\"token punctuation\">)</span></code></pre></div>\n<p><span\n      class=\"gatsby-resp-image-wrapper\"\n      style=\"position: relative; display: block; margin-left: auto; margin-right: auto;  max-width: 383px;\"\n    >\n      <a\n    class=\"gatsby-resp-image-link\"\n    href=\"/static/d110bc725bc2a706d9ab7ac80e68eeb1/b4d9b/index_20_0.png\"\n    style=\"display: block\"\n    target=\"_blank\"\n    rel=\"noopener\"\n  >\n    <span\n    class=\"gatsby-resp-image-background-image\"\n    style=\"padding-bottom: 69.71279373368147%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAOCAYAAAAvxDzwAAAACXBIWXMAAAsSAAALEgHS3X78AAACQUlEQVQ4y42U627TQBCF8/5PgfgBQkICCURLSVtB0zSlLRAlhSbkZqfxJV5fd51ry2FmXEcJVIhIG429M9+cmZ11BY/8oiiC4zhwXRe+72M8tuF5Hmzb3jxb1oiWhSRJHqJ+yX9lNpvJS2OWcPwUWTYnoMZ0mkCpFCGtoeXDcUME01jeOU5ACRTBI4ShlpgkmSHPF6gYY5AS0J1mOKj3CZTJCgItKwoN6l9v0R0qxFEue0oZAhnZL/09L0Uc56jM53PMcgNvqlFtDKG2nIIgE+DpFxvdQSD29l5pK1XEJAkB8zwnhTE8AlXPBjuO/wbqv2wBcg+1zgqFfwCnAtT/BfSp/xuFjwHZiUuJo0LhT+phCWSfskwpmftdKtSaTymFv1Uyr7NvY+yf9tHu+mIf1AcSHBLo6tqBNY7ELqGN5i0UJRSFudE7Cjk7w56+beOcHBn4snpTnCQp/nhhoTdS0g6G+TIhlJBEydhkaboLpI0PZD/bu8ZFayLAV0cdAXLZJ1cW+qNwo5CBhzQhAtQMfCj58Hwo6tix2hjg+f53XLYnUs7r4440nhXWqKcDKxR42fMjig2UKMxhqI+un2Gv1iuaTuvdSQ9P3rREXY0UvXj/g65iIsmOP4/kkDg5J/EollvkUVwly3g0FJJsQc2eUPk8RnM0Ox4+cWl2hB6tenNMN8FQvxe4oRGaeHRd9UKuHb9vdT3EFFtZr9dYrVZysQ2NDz/z7/5uTcHZ5oPBB7dcLovPwP0dQaLNniZRq+VC7N8TwhHZ1Y8tVAAAAABJRU5ErkJggg=='); background-size: cover; display: block;\"\n  ></span>\n  <img\n        class=\"gatsby-resp-image-image\"\n        alt=\"png\"\n        title=\"png\"\n        src=\"/static/d110bc725bc2a706d9ab7ac80e68eeb1/b4d9b/index_20_0.png\"\n        srcset=\"/static/d110bc725bc2a706d9ab7ac80e68eeb1/1abb1/index_20_0.png 250w,\n/static/d110bc725bc2a706d9ab7ac80e68eeb1/b4d9b/index_20_0.png 383w\"\n        sizes=\"(max-width: 383px) 100vw, 383px\"\n        loading=\"lazy\"\n      />\n  </a>\n    </span></p>\n<h1>Total dataset summary stats</h1>\n<div class=\"gatsby-highlight\" data-language=\"python\"><pre class=\"language-python\"><code class=\"language-python\"><span class=\"token comment\"># Note: specimen 741428906 is in both the training and testing datasets.</span>\n<span class=\"token comment\"># This next line will keep the testing one, with prefix = 'TEST_DATA_SET/741428906/'.</span>\n<span class=\"token comment\"># I.e. the training version of 741428906 is dropped from the manifest. We only</span>\n<span class=\"token comment\"># have 10 test neurons, don't want to lose one. Although not much of a test if</span>\n<span class=\"token comment\"># the answers are in the test question.</span>\nall_specimens <span class=\"token operator\">=</span> <span class=\"token punctuation\">{</span> <span class=\"token operator\">**</span>training_neurons<span class=\"token punctuation\">,</span> <span class=\"token operator\">**</span> testing_neurons<span class=\"token punctuation\">}</span>\n\nbytes_accum <span class=\"token operator\">=</span> <span class=\"token number\">0</span>\n<span class=\"token keyword\">for</span> specimen_name <span class=\"token keyword\">in</span> all_specimens<span class=\"token punctuation\">:</span>\n  specimen <span class=\"token operator\">=</span> all_specimens<span class=\"token punctuation\">[</span>specimen_name<span class=\"token punctuation\">]</span>\n  bytes_accum <span class=\"token operator\">+=</span> specimen<span class=\"token punctuation\">[</span><span class=\"token string\">\"size\"</span><span class=\"token punctuation\">]</span>\n  <span class=\"token comment\"># TODO: there must be a more elegant way to reduce an array in Python</span>\n\n<span class=\"token keyword\">print</span><span class=\"token punctuation\">(</span><span class=\"token string\">\"Total bytes: %s\"</span> <span class=\"token operator\">%</span> bytes_accum<span class=\"token punctuation\">)</span>\n\n\ngrand_total_file_size<span class=\"token punctuation\">,</span> grand_total_file_unit <span class=\"token operator\">=</span> format_bytes<span class=\"token punctuation\">(</span>bytes_accum<span class=\"token punctuation\">)</span>\n<span class=\"token keyword\">print</span><span class=\"token punctuation\">(</span><span class=\"token string\">\"Number of cells in dataset manifest = %d\"</span> <span class=\"token operator\">%</span> <span class=\"token builtin\">len</span><span class=\"token punctuation\">(</span>all_specimens<span class=\"token punctuation\">)</span><span class=\"token punctuation\">)</span>\n<span class=\"token keyword\">print</span><span class=\"token punctuation\">(</span><span class=\"token string\">\"Total size of training dataset = \"</span> <span class=\"token operator\">+</span> <span class=\"token string\">'{:4.1f}'</span><span class=\"token punctuation\">.</span><span class=\"token builtin\">format</span><span class=\"token punctuation\">(</span>grand_total_file_size<span class=\"token punctuation\">)</span> <span class=\"token operator\">+</span> <span class=\"token string\">\" \"</span> <span class=\"token operator\">+</span> grand_total_file_unit <span class=\"token operator\">+</span> <span class=\"token string\">\" (\"</span> <span class=\"token operator\">+</span> <span class=\"token builtin\">str</span><span class=\"token punctuation\">(</span>bytes_accum<span class=\"token punctuation\">)</span> <span class=\"token operator\">+</span> <span class=\"token string\">\" bytes)\"</span><span class=\"token punctuation\">)</span></code></pre></div>\n<div class=\"gatsby-highlight\" data-language=\"text\"><pre class=\"language-text\"><code class=\"language-text\">Total bytes: 2671458682941\nNumber of cells in dataset manifest = 114\nTotal size of training dataset =  2.4 TB (2671458682941 bytes)</code></pre></div>\n<div class=\"gatsby-highlight\" data-language=\"python\"><pre class=\"language-python\"><code class=\"language-python\"><span class=\"token comment\"># Double check those numbers: just total every single object</span>\ntotal_bytes_for_all_objects <span class=\"token operator\">=</span> <span class=\"token number\">0</span>\n<span class=\"token keyword\">for</span> s3_object <span class=\"token keyword\">in</span> bucket<span class=\"token punctuation\">.</span>objects<span class=\"token punctuation\">.</span><span class=\"token builtin\">all</span><span class=\"token punctuation\">(</span><span class=\"token punctuation\">)</span><span class=\"token punctuation\">:</span>\n  total_bytes_for_all_objects <span class=\"token operator\">+=</span> s3_object<span class=\"token punctuation\">.</span>size\n  \nrecheck_size<span class=\"token punctuation\">,</span> recheck_unit <span class=\"token operator\">=</span> format_bytes<span class=\"token punctuation\">(</span>total_bytes_for_all_objects<span class=\"token punctuation\">)</span>\n<span class=\"token keyword\">print</span><span class=\"token punctuation\">(</span><span class=\"token string\">\"Total size of all files in dataset = \"</span> <span class=\"token operator\">+</span> <span class=\"token string\">'{:4.2f}'</span><span class=\"token punctuation\">.</span><span class=\"token builtin\">format</span><span class=\"token punctuation\">(</span>recheck_size<span class=\"token punctuation\">)</span> <span class=\"token operator\">+</span> <span class=\"token string\">\" \"</span> <span class=\"token operator\">+</span> recheck_unit <span class=\"token operator\">+</span> <span class=\"token string\">\" (\"</span> <span class=\"token operator\">+</span> <span class=\"token builtin\">str</span><span class=\"token punctuation\">(</span>total_bytes_for_all_objects<span class=\"token punctuation\">)</span> <span class=\"token operator\">+</span> <span class=\"token string\">\" bytes)\"</span><span class=\"token punctuation\">)</span></code></pre></div>\n<div class=\"gatsby-highlight\" data-language=\"text\"><pre class=\"language-text\"><code class=\"language-text\">Total size of all files in dataset = 2.47 TB (2713810170427 bytes)</code></pre></div>\n<h2>Write specimens_manifest.json</h2>\n<p>The rest of the notebooks in this project make use of <code class=\"language-text\">specimens_manifest.json</code> which is just a semantically organized manifest of all the files in the dataset, organized by specimen, as image stack, catalogued by specimen<em>id. The head of `specimens</em>manifest.json` looks like:</p>\n<div class=\"gatsby-highlight\" data-language=\"javascript\"><pre class=\"language-javascript\"><code class=\"language-javascript\"><span class=\"token punctuation\">{</span>\n    <span class=\"token string\">\"647225829\"</span><span class=\"token punctuation\">:</span> <span class=\"token punctuation\">{</span>\n        <span class=\"token string\">\"id\"</span><span class=\"token punctuation\">:</span> <span class=\"token string\">\"647225829\"</span><span class=\"token punctuation\">,</span>\n        <span class=\"token string\">\"bucket_prefix\"</span><span class=\"token punctuation\">:</span> <span class=\"token string\">\"647225829/\"</span><span class=\"token punctuation\">,</span>\n        <span class=\"token string\">\"swc\"</span><span class=\"token punctuation\">:</span> <span class=\"token string\">\"647225829/647225829.swc\"</span><span class=\"token punctuation\">,</span>\n        <span class=\"token string\">\"bytes\"</span><span class=\"token punctuation\">:</span> <span class=\"token number\">26559180540</span><span class=\"token punctuation\">,</span>\n        <span class=\"token string\">\"image_stack\"</span><span class=\"token punctuation\">:</span> <span class=\"token punctuation\">[</span>\n            <span class=\"token string\">\"647225829/reconstruction_0_0539044533_639893239-0001.tif\"</span><span class=\"token punctuation\">,</span>\n            <span class=\"token string\">\"647225829/reconstruction_0_0539044533_639893239-0002.tif\"</span><span class=\"token punctuation\">,</span></code></pre></div>\n<p>The file specimens_manifest.json is a logical view of the (~6K) physical files in the dataset. File names within the manifest are relative to the root of the dataset. This file can be used later to provide a clean interface to the library of specimens as well as maintain a per-specimen download cache (useful for notebooks that only process a single specimen because of file system size limitations). Having a download cache is very handy to speed up repeated notebook <code class=\"language-text\">Runtime | Run all</code> because each specimen’s data is 6GB to 60GB in size, which is boring to watch happen repeatedly unneccessarily </p>\n<p>The specimens in the manifest JSON are listed in a flat dictionary, keyed by specimen ID. Filenames in the manifest are relative to the root of the bucket where the specimen dataset resides. </p>\n<p>(Note by file naming relative to the root of the original source dataset bucket (rather than Colab file system absolute names) folks could also use the specimens_manifest.json file outside the context of Colab. It is a reusable convenience for experimentation on other platforms.)</p>\n<p>Each specimen has two properties, the local full filename to the .swc file (if any), and the array of full local filenames to the TIFF files in the z-stack.</p>\n<p>The contents of specimens_manifest.json plus the directory name of root of the local file system cache of files from the dataset is sufficient to resolve to full file names of specimens files, with all the data corralling hassles already taken care of for code that actually does something with these files.</p>\n<p>Might as well list the specimens sorted by size, smallest first. This way a casual tire kicker will grab the easiest/smallest specimen first. And files might as well be listed sorted alphbetically, which Python APIs do not guarantee.</p>\n<p>Note: a copy of <code class=\"language-text\">specimens_manifest.json</code> is stored on reconstrue.com. This is used by default by other notebooks in this project. That file was created by the following code cell:</p>\n<div class=\"gatsby-highlight\" data-language=\"python\"><pre class=\"language-python\"><code class=\"language-python\"><span class=\"token comment\"># Goal: write specimens_manifest.json</span>\nspecimens_manifest <span class=\"token operator\">=</span> <span class=\"token punctuation\">{</span><span class=\"token punctuation\">}</span>\n\n<span class=\"token comment\"># Set up data_dir, where to write to:</span>\ndata_dir <span class=\"token operator\">=</span> <span class=\"token string\">\"/content/brightfield_data/\"</span>\n<span class=\"token keyword\">if</span> <span class=\"token keyword\">not</span> os<span class=\"token punctuation\">.</span>path<span class=\"token punctuation\">.</span>isdir<span class=\"token punctuation\">(</span>data_dir<span class=\"token punctuation\">)</span><span class=\"token punctuation\">:</span>\n  os<span class=\"token punctuation\">.</span>mkdir<span class=\"token punctuation\">(</span>data_dir<span class=\"token punctuation\">)</span>\nmanifest_file_name <span class=\"token operator\">=</span> os<span class=\"token punctuation\">.</span>path<span class=\"token punctuation\">.</span>join<span class=\"token punctuation\">(</span>data_dir<span class=\"token punctuation\">,</span> <span class=\"token string\">\"specimens_manifest.json\"</span><span class=\"token punctuation\">)</span>\n\n<span class=\"token keyword\">for</span> specimen_name <span class=\"token keyword\">in</span> all_specimens<span class=\"token punctuation\">:</span>\n  specimen <span class=\"token operator\">=</span> all_specimens<span class=\"token punctuation\">[</span>specimen_name<span class=\"token punctuation\">]</span>\n  specimens_manifest<span class=\"token punctuation\">[</span>specimen_name<span class=\"token punctuation\">]</span> <span class=\"token operator\">=</span> <span class=\"token punctuation\">{</span>\n    <span class=\"token string\">\"id\"</span><span class=\"token punctuation\">:</span> specimen_name<span class=\"token punctuation\">,</span>\n    <span class=\"token string\">\"bucket_prefix\"</span><span class=\"token punctuation\">:</span> specimen<span class=\"token punctuation\">[</span><span class=\"token string\">\"prefix\"</span><span class=\"token punctuation\">]</span><span class=\"token punctuation\">,</span>\n    <span class=\"token string\">\"swc\"</span><span class=\"token punctuation\">:</span> specimen<span class=\"token punctuation\">[</span><span class=\"token string\">\"swc\"</span><span class=\"token punctuation\">]</span><span class=\"token punctuation\">,</span>\n    <span class=\"token string\">\"bytes\"</span><span class=\"token punctuation\">:</span> specimen<span class=\"token punctuation\">[</span><span class=\"token string\">\"size\"</span><span class=\"token punctuation\">]</span><span class=\"token punctuation\">,</span>\n    <span class=\"token string\">\"image_stack\"</span><span class=\"token punctuation\">:</span> specimen<span class=\"token punctuation\">[</span><span class=\"token string\">\"imagestack\"</span><span class=\"token punctuation\">]</span>\n  <span class=\"token punctuation\">}</span> \n  <span class=\"token comment\"># fields: {\"imagestack\": imagestack, \"size\": imagestack_bytes}</span>\n\n<span class=\"token keyword\">with</span> <span class=\"token builtin\">open</span><span class=\"token punctuation\">(</span>manifest_file_name<span class=\"token punctuation\">,</span> <span class=\"token string\">\"w+\"</span><span class=\"token punctuation\">)</span> <span class=\"token keyword\">as</span> mani<span class=\"token punctuation\">:</span>\n  json<span class=\"token punctuation\">.</span>dump<span class=\"token punctuation\">(</span>specimens_manifest<span class=\"token punctuation\">,</span> mani<span class=\"token punctuation\">)</span></code></pre></div>\n<h1>Appendix #1: The curious case of specimen 741428906</h1>\n<p>Looks like 741428906 got into both the training and test datasets.</p>\n<div class=\"gatsby-highlight\" data-language=\"python\"><pre class=\"language-python\"><code class=\"language-python\"><span class=\"token comment\"># Notice how len(all_specimens) &lt; len(training_neurons) + len(testing_neurons)</span>\n<span class=\"token comment\"># There seems to be one missing</span>\n<span class=\"token keyword\">print</span><span class=\"token punctuation\">(</span><span class=\"token builtin\">len</span><span class=\"token punctuation\">(</span>training_neurons<span class=\"token punctuation\">)</span><span class=\"token punctuation\">)</span>\n<span class=\"token keyword\">print</span><span class=\"token punctuation\">(</span><span class=\"token builtin\">len</span><span class=\"token punctuation\">(</span>testing_neurons<span class=\"token punctuation\">)</span><span class=\"token punctuation\">)</span>\n<span class=\"token keyword\">print</span><span class=\"token punctuation\">(</span><span class=\"token builtin\">len</span><span class=\"token punctuation\">(</span>all_specimens<span class=\"token punctuation\">)</span><span class=\"token punctuation\">)</span>\n\n<span class=\"token comment\"># Notice how 741428906 is in both training and test subsets</span>\naSet <span class=\"token operator\">=</span> <span class=\"token builtin\">set</span><span class=\"token punctuation\">(</span>training_neurons<span class=\"token punctuation\">)</span>\nbSet <span class=\"token operator\">=</span> <span class=\"token builtin\">set</span><span class=\"token punctuation\">(</span>testing_neurons<span class=\"token punctuation\">)</span>\n<span class=\"token keyword\">for</span> name <span class=\"token keyword\">in</span> aSet<span class=\"token punctuation\">.</span>intersection<span class=\"token punctuation\">(</span>bSet<span class=\"token punctuation\">)</span><span class=\"token punctuation\">:</span>\n    <span class=\"token keyword\">print</span><span class=\"token punctuation\">(</span>name<span class=\"token punctuation\">,</span> all_specimens<span class=\"token punctuation\">[</span>name<span class=\"token punctuation\">]</span><span class=\"token punctuation\">)</span></code></pre></div>\n<div class=\"gatsby-highlight\" data-language=\"text\"><pre class=\"language-text\"><code class=\"language-text\">105\n10\n114\n741428906 {&#39;prefix&#39;: &#39;TEST_DATA_SET/741428906/&#39;, &#39;swc&#39;: None, &#39;imagestack&#39;: [&#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0001.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0002.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0003.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0004.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0005.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0006.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0007.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0008.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0009.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0010.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0011.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0012.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0013.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0014.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0015.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0016.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0017.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0018.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0019.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0020.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0021.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0022.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0023.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0024.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0025.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0026.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0027.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0028.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0029.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0030.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0031.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0032.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0033.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0034.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0035.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0036.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0037.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0038.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0039.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0040.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0041.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0042.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0043.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0044.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0045.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0046.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0047.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0048.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0049.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0050.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0051.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0052.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0053.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0054.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0055.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0056.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0057.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0058.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0059.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0060.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0061.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0062.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0063.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0064.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0065.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0066.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0067.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0068.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0069.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0070.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0071.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0072.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0073.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0074.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0075.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0076.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0077.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0078.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0079.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0080.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0081.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0082.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0083.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0084.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0085.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0086.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0087.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0088.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0089.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0090.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0091.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0092.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0093.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0094.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0095.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0096.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0097.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0098.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0099.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0100.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0101.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0102.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0103.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0104.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0105.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0106.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0107.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0108.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0109.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0110.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0111.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0112.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0113.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0114.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0115.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0116.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0117.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0118.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0119.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0120.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0121.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0122.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0123.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0124.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0125.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0126.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0127.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0128.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0129.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0130.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0131.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0132.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0133.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0134.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0135.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0136.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0137.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0138.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0139.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0140.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0141.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0142.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0143.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0144.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0145.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0146.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0147.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0148.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0149.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0150.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0151.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0152.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0153.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0154.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0155.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0156.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0157.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0158.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0159.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0160.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0161.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0162.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0163.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0164.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0165.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0166.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0167.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0168.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0169.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0170.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0171.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0172.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0173.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0174.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0175.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0176.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0177.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0178.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0179.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0180.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0181.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0182.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0183.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0184.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0185.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0186.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0187.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0188.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0189.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0190.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0191.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0192.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0193.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0194.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0195.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0196.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0197.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0198.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0199.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0200.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0201.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0202.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0203.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0204.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0205.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0206.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0207.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0208.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0209.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0210.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0211.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0212.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0213.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0214.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0215.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0216.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0217.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0218.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0219.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0220.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0221.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0222.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0223.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0224.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0225.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0226.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0227.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0228.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0229.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0230.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0231.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0232.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0233.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0234.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0235.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0236.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0237.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0238.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0239.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0240.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0241.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0242.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0243.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0244.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0245.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0246.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0247.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0248.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0249.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0250.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0251.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0252.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0253.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0254.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0255.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0256.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0257.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0258.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0259.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0260.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0261.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0262.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0263.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0264.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0265.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0266.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0267.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0268.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0269.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0270.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0271.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0272.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0273.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0274.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0275.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0276.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0277.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0278.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0279.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0280.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0281.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0282.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0283.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0284.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0285.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0286.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0287.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0288.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0289.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0290.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0291.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0292.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0293.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0294.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0295.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0296.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0297.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0298.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0299.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0300.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0301.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0302.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0303.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0304.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0305.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0306.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0307.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0308.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0309.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0310.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0311.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0312.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0313.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0314.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0315.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0316.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0317.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0318.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0319.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0320.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0321.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0322.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0323.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0324.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0325.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0326.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0327.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0328.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0329.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0330.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0331.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0332.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0333.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0334.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0335.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0336.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0337.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0338.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0339.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0340.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0341.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0342.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0343.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0344.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0345.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0346.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0347.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0348.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0349.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0350.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0351.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0352.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0353.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0354.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0355.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0356.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0357.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0358.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0359.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0360.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0361.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0362.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0363.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0364.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0365.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0366.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0367.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0368.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0369.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0370.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0371.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0372.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0373.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0374.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0375.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0376.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0377.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0378.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0379.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0380.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0381.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0382.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0383.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0384.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0385.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0386.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0387.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0388.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0389.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0390.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0391.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0392.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0393.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0394.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0395.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0396.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0397.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0398.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0399.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0400.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0401.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0402.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0403.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0404.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0405.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0406.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0407.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0408.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0409.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0410.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0411.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0412.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0413.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0414.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0415.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0416.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0417.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0418.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0419.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0420.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0421.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0422.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0423.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0424.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0425.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0426.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0427.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0428.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0429.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0430.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0431.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0432.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0433.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0434.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0435.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0436.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0437.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0438.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0439.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0440.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0441.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0442.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0443.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0444.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0445.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0446.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0447.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0448.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0449.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0450.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0451.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0452.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0453.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0454.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0455.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0456.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0457.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0458.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0459.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0460.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0461.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0462.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0463.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0464.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0465.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0466.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0467.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0468.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0469.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0470.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0471.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0472.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0473.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0474.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0475.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0476.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0477.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0478.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0479.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0480.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0481.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0482.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0483.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0484.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0485.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0486.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0487.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0488.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0489.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0490.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0491.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0492.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0493.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0494.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0495.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0496.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0497.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0498.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0499.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0500.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0501.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0502.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0503.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0504.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0505.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0506.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0507.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0508.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0509.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0510.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0511.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0512.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0513.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0514.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0515.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0516.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0517.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0518.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0519.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0520.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0521.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0522.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0523.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0524.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0525.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0526.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0527.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0528.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0529.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0530.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0531.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0532.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0533.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0534.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0535.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0536.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0537.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0538.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0539.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0540.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0541.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0542.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0543.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0544.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0545.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0546.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0547.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0548.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0549.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0550.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0551.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0552.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0553.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0554.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0555.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0556.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0557.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0558.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0559.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0560.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0561.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0562.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0563.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0564.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0565.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0566.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0567.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0568.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0569.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0570.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0571.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0572.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0573.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0574.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0575.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0576.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0577.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0578.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0579.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0580.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0581.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0582.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0583.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0584.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0585.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0586.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0587.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0588.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0589.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0590.tif&#39;, &#39;TEST_DATA_SET/741428906/reconstruction_0_0500371379_714485370-0591.tif&#39;], &#39;size&#39;: 42301483965}</code></pre></div>\n<div class=\"gatsby-highlight\" data-language=\"python\"><pre class=\"language-python\"><code class=\"language-python\"><span class=\"token comment\"># Check the file on the file system.</span>\n<span class=\"token comment\"># On the file system, specimens_manifest.json is a bit long for display (> 5K lines). </span>\n<span class=\"token comment\"># So, here's the first 20 lines; the rest is similar.</span>\n!python <span class=\"token operator\">-</span>m json<span class=\"token punctuation\">.</span>tool <span class=\"token punctuation\">{</span>manifest_file_name<span class=\"token punctuation\">}</span> <span class=\"token punctuation\">{</span>manifest_file_name<span class=\"token punctuation\">}</span><span class=\"token string\">\".pretty.json\"</span>\n!echo <span class=\"token punctuation\">{</span>data_dir<span class=\"token punctuation\">}</span>\n!ls <span class=\"token operator\">-</span>l <span class=\"token punctuation\">{</span>data_dir<span class=\"token punctuation\">}</span>\n!head <span class=\"token operator\">-</span><span class=\"token number\">20</span> <span class=\"token punctuation\">{</span>manifest_file_name<span class=\"token punctuation\">}</span><span class=\"token string\">\".pretty.json\"</span></code></pre></div>\n<div class=\"gatsby-highlight\" data-language=\"text\"><pre class=\"language-text\"><code class=\"language-text\">/content/brightfield_data/\ntotal 7012\n-rw-r--r-- 1 root root 3266153 Nov 18 10:29 specimens_manifest.json\n-rw-r--r-- 1 root root 3910364 Nov 18 10:29 specimens_manifest.json.pretty.json\n{\n    &quot;647225829&quot;: {\n        &quot;id&quot;: &quot;647225829&quot;,\n        &quot;bucket_prefix&quot;: &quot;647225829/&quot;,\n        &quot;swc&quot;: &quot;647225829/647225829.swc&quot;,\n        &quot;bytes&quot;: 26559180540,\n        &quot;image_stack&quot;: [\n            &quot;647225829/reconstruction_0_0539044533_639893239-0001.tif&quot;,\n            &quot;647225829/reconstruction_0_0539044533_639893239-0002.tif&quot;,\n            &quot;647225829/reconstruction_0_0539044533_639893239-0003.tif&quot;,\n            &quot;647225829/reconstruction_0_0539044533_639893239-0004.tif&quot;,\n            &quot;647225829/reconstruction_0_0539044533_639893239-0005.tif&quot;,\n            &quot;647225829/reconstruction_0_0539044533_639893239-0006.tif&quot;,\n            &quot;647225829/reconstruction_0_0539044533_639893239-0007.tif&quot;,\n            &quot;647225829/reconstruction_0_0539044533_639893239-0008.tif&quot;,\n            &quot;647225829/reconstruction_0_0539044533_639893239-0009.tif&quot;,\n            &quot;647225829/reconstruction_0_0539044533_639893239-0010.tif&quot;,\n            &quot;647225829/reconstruction_0_0539044533_639893239-0011.tif&quot;,\n            &quot;647225829/reconstruction_0_0539044533_639893239-0012.tif&quot;,\n            &quot;647225829/reconstruction_0_0539044533_639893239-0013.tif&quot;,</code></pre></div>\n<div class=\"gatsby-highlight\" data-language=\"python\"><pre class=\"language-python\"><code class=\"language-python\"><span class=\"token comment\"># To download specimens_manifest.json</span>\n<span class=\"token comment\">#</span>\n<span class=\"token comment\"># from google.colab import files</span>\n<span class=\"token comment\"># files.download(manifest_file_name)</span></code></pre></div>","frontmatter":{"title":"Brightfield Challenge Dataset Manifest","date":"September 20, 2019","description":"TODO: a discription for cover card"}}},"pageContext":{"isCreatedByStatefulCreatePages":false,"slug":"/brightfield_challenge_data_manifest/","previous":{"fields":{"slug":"/brightfield_reconstruction_challenge/"},"frontmatter":{"title":"Brightfield Auto-Reconstruction Challenge"}},"next":{"fields":{"slug":"/jupyter_book_to_colab/"},"frontmatter":{"title":"Jupyter Book to Colab"}}}}}