TASK LIST

(1) Method: discoverIndexes(yearsHistory=7)

Discover complete set of EDGAR "full" index files
 - top-level represents current quarter.
 - otherwise broken into historical quarters.
 - Use index.json? (sitemap.xml has not been updated since 2012)
 - Ignore daily index files for now

Create queue of Requests to fetch all full-index/YYYY/QTRN/index.json files
Fetch with progress reporting.
Retry on failure.

Fetch indexes in reverse annual order. Fetch no more than 7 yrs by default.
(Yrs) * (4 qtr dirs + 1 yr dir) + 1 top level dir
= 36 HTTPS requests

Fetching all historical full-index/index.json files will require 116 HTTPS requests - unnecesarily slow.

(1.1) Improve discoverIndexes to cache raw file content. Do not parse JSON before saving to cache.

(1.2) Improve discoverIndexes with progress state tracking: to make it restartable and so we can determine when it did not complete successfully.

(2) Method: listIndexes({type=(master|form|company|xbrl|none)})

Returns: list of known .gz index files retrieved above in _reverse chronological order_. Ignore file entries with extensions other than .gz. (.idx, .Z, .xml)

(3) Retrieve compressed index files:
 - (company.gz - sorted by company name)
 - (form.gz - sorted by form type)
 - master.gz - sorted by CIK number
 - (xbrl.gz - sorted by CIK number)

Expose search interface for each index type:
 - grep over raw text buffer (do not parse or copy!)
 - result should show N lines of context

Parse file lines (do this last)