# Module: dataBroker

Contains all of the lower level data processing functions, and also acts as an interface for calling the fileOperations.

Author:
  - Seth Hollingsead
Copyright:
  - Copyright © 2022-… by Seth Hollingsead. All rights reserved
Source:
  - brokers/dataBroker.js, line 1

### Requires

- module:ruleBroker
- module:configurator
- module:loggers
- module:data
- @haystacks/constants
- path

### Methods

#### (inner) addConstantsValidationData(constantLibraryData) → {void}

Adds a library of constants vaidation data to the existing constants vaidation data.

##### Parameters:

Name | Type | Description
--- | --- | ---
`constantLibraryData` | array.<array.<string, object>> | The array of data that should be added to the validation data set for the purpose of validation.

Author:
  - Seth Hollingsead
Source:
  - brokers/dataBroker.js, line 649

##### Returns:

Type
  void

#### (inner) addDeeplyNestedConstantsValidationData(contextName, deeplyNestedData) → {void}

Adds all the constants validation auxiliary data that is deeply nested inside sub-data structures to the main D-data structure. Such as file paths, and validation messages.

##### Parameters:

Name | Type | Description
--- | --- | ---
`contextName` | string | The name that should be used when accessing and also adding the data to the D-data structure.
`deeplyNestedData` | array.<array.<string, object>> | The actual data that should be added.

Author:
  - Seth Hollingsead
Source:
  - brokers/dataBroker.js, line 681

##### Returns:

Type
  void

#### (inner) clearData(dataStorageContextName) → {void}

Clears out all of the data stored in the DataStorage data hive of the D data structure, or a particular stored data element if a contextName is provided.

##### Parameters:

Name | Type | Description
--- | --- | ---
`dataStorageContextName` | string | (OPTIONAL) The sub-data storage hive which should be cleared.

Author:
  - Seth Hollingsead
Source:
  - brokers/dataBroker.js, line 607

##### Returns:

Type
  void

#### (inner) determineMergeTarget(targetData, dataToMerge) → {string}

Scans the target and source data objects recursively and determines what level of the target the source should be merged at.

##### Parameters:

Name | Type | Description
--- | --- | ---
`targetData` | object | The target data object where the data should in theory be merged with for the purpose of this simulated merge.
`dataToMerge` | object | The data that should be merged in this simulated merge scenario.

Author:
  - Seth Hollingsead
Source:
  - brokers/dataBroker.js, line 819

##### Returns:

The string of the namespace where the dataToMerge should be merged with the target data object.

Type
  string

#### (inner) extractDataFromPapaParseObject(data, contextName) → {object}

Extracts the parsed data from the papa parse data object.

##### Parameters:

Name | Type | Description
--- | --- | ---
`data` | object | Contains the full contents of the papa parse data object, for which we need to get data out of.
`contextName` | string | The name of the context either Command, Macro from which data should be retrieved.

Author:
  - Seth Hollingsead
Source:
  - brokers/dataBroker.js, line 750

##### Returns:

The fully parsed data that we intend to use for the application.

Type
  object

#### (inner) findIndividualDebugConfigSetting(filesToLoad) → {boolean}

Finds if a debugSetting is set for a particular set of config files.

##### Parameters:

Name | Type | Description
--- | --- | ---
`filesToLoad` | array.<string> | A list of files that should be searched for a system config file and then for a debugSetting in that file.

Author:
  - Seth Hollingsead
Source:
  - brokers/dataBroker.js, line 87

##### Returns:

A True or False value to indicate what the value of the debug setting is for the set of system config files.

Type
  boolean

#### (inner) findUniversalDebugConfigSetting(appConfigFilesToLoad, frameworkConfigFilesToLoad) → {boolean}

Determines if there is any True setting for the debug settings in either the application system config or the framework system config files.

##### Parameters:

Name | Type | Description
--- | --- | ---
`appConfigFilesToLoad` | array.<string> | The list of files for the app config files.
`frameworkConfigFilesToLoad` | array.<string> | The list of files for the framework config files.

Author:
  - Seth Hollingsead
Source:
  - brokers/dataBroker.js, line 58

##### Returns:

A True or False value to indicate if debug setting value is set in either the app config system file or the framework config system file.

Type
  boolean

#### (inner) getData(dataStorageContextName) → {object}

Gets some data from a caller specified sub-data storage hive name.

##### Parameters:

Name | Type | Description
--- | --- | ---
`dataStorageContextName` | string | The sub-data storage hive which should be retrieved.

Author:
  - Seth Hollingsead
Source:
  - brokers/dataBroker.js, line 585

##### Returns:

the data that is found, if any at the specified location on the data storage hive.

Type
  object

#### (inner) getDataCategoryDetailNameFromContextName(contextName) → {string}

Gets the data category detail name, given the context name.

##### Parameters:

Name | Type | Description
--- | --- | ---
`contextName` | string | The name which will be something like Command_ApiPost or Script_ApiMacroX.

Author:
  - Seth Hollingsead
Source:
  - brokers/dataBroker.js, line 728

##### Returns:

The string after the first cUnderscore (ApiPost or ApiMacroX).

Type
  string

#### (inner) getDataCategoryFromContextName(contextName) → {string}

Gets the data category, given the context name.

##### Parameters:

Name | Type | Description
--- | --- | ---
`contextName` | string | The context name which will be something like Application_xxxx or Script_nnnn or Command_yyyy

Author:
  - Seth Hollingsead
Source:
  - brokers/dataBroker.js, line 707

##### Returns:

The string before the first cUnderscore (Application, Script, Command, etc).

Type
  string

#### (inner) getDataElement(dataObject, pageName, elementName) → {string}

Gets the named value from the data object then cleans it from any carriage return characters.

##### Parameters:

Name | Type | Description
--- | --- | ---
`dataObject` | object | The data object with all data.
`pageName` | string | The name of the page where the data should be found.
`elementName` | string | The name of the data element that should be found for the given page.

Author:
  - Seth Hollingsead
Source:
  - brokers/dataBroker.js, line 964

##### Returns:

The data element with all carriage return characters removed from it.

Type
  string

#### (inner) getDataElementCount(dataObject, pageName, elementNamePattern) → {integer}

Gets the count of the number of elements that match a given patern, if an empty string is passed in for the pattern then the count of the collection is returned with no pattern matching.

##### Parameters:

Name | Type | Description
--- | --- | ---
`dataObject` | object | The data map with all data.
`pageName` | string | The name of the page were the data shoudl be found.
`elementNamePattern` | string | A string containing a pattern that should be matched in the object collection, to establish a count of elements that match this pattern.

Author:
  - Seth Hollingsead
Source:
  - brokers/dataBroker.js, line 994

##### Returns:

A count of either the number of data elements in the object collection, or a count for the number of data elements that match a pattern that is passed in.

Type
  integer

#### (inner) initializeConstantsValidationData() → {void}

Initializes the constants validation data structure.

Author:
  - Seth Hollingsead
Source:
  - brokers/dataBroker.js, line 629

##### Returns:

Type
  void

#### (inner) loadAllCsvData(filesToLoad, contextName) → {object}

Loads al of the contents of all files and folders and sub-folders at the specified path and builds a list of files to load, then loads them accordingly in the D.contextName_fileName.

##### Parameters:

Name | Type | Description
--- | --- | ---
`filesToLoad` | array.<string> | The data structure containing all of the files to load data from.
`contextName` | string | The context name that should be used when adding data to the D data structure.

Author:
  - Seth Hollingsead
Source:
  - brokers/dataBroker.js, line 131

##### Returns:

The data in a JSON object after it was loaded from the file.

Type
  object

#### (inner) loadAllJsonData(filesToLoad, contextName) → {object}

Loads all the contents of all files and folders and sub-folders at the specified path and builds a list of files to load, then loads them accordingly in the D.contextName.

##### Parameters:

Name | Type | Description
--- | --- | ---
`filesToLoad` | array.<string> | The data structure containing all of the files to load data from.
`contextName` | string | The context name that should be used when adding data to the D-data structure.

Author:
  - Seth Hollingsead
Source:
  - brokers/dataBroker.js, line 255

##### Returns:

A JSON object that contains all fo the data that was loaded and parsed from all the input files list.

Type
  object

#### (inner) loadedAllXmlData(filesToLoad, contextName) → {object}

Loads all the context of all files and folders and sub-folders at the specified path and builds a list of files to load, then loads them accordingly in the D.contextName_fileName.

##### Parameters:

Name | Type | Description
--- | --- | ---
`filesToLoad` | array.<string> | The data structure containing all of the files to load data from.
`contextName` | string | The context name that should be used when adding data to the D data structure.

Author:
  - Seth Hollingsead
Source:
  - brokers/dataBroker.js, line 180

##### Returns:

A JSON object that contains all of the data that was loaded and parsed from all the input files list.

Type
  object

#### (inner) mergeData(targetData, dataCategory, pageName, dataToMerge) → {object}

Merge data with the D data structure for the specified data category and optional name.

##### Parameters:

Name | Type | Description
--- | --- | ---
`targetData` | object | The target data object where the dataToMerge should be merged with.
`dataCategory` | string | Command or Script to indicate what category the test data should be used as.
`pageName` | string | (Optional) The name of the page where the data should be merged under. Pass as empty string if nothing.
`dataToMerge` | object | The data to be merged.

Author:
  - Seth Hollingsead
Source:
  - brokers/dataBroker.js, line 879

##### Returns:

A merged set of data combining all of the original data plus all of the additional data from the dataToMerge data set.

Type
  object

#### (inner) preprocessJsonFile(fileToLoad) → {object}

Load all of the data from a single JSON data file.

##### Parameters:

Name | Type | Description
--- | --- | ---
`fileToLoad` | string | The fully qualified path to the file that should be loaded.

Author:
  - Seth Hollingsead
Source:
  - brokers/dataBroker.js, line 500

##### Returns:

The JSON data object that was loaded from the specified JSON data file.

Type
  object

#### (inner) processCsvData(data, contextName) → {object}

Processes all of the CSV data into a usable format and executes any additional processing rules.

##### Parameters:

Name | Type | Description
--- | --- | ---
`data` | object | A JSON object that contains all of the data loaded from a CSV file.
`contextName` | string | The name that should be used when adding the objects to the D data structure for data-sharing.

Author:
  - Seth Hollingsead
Source:
  - brokers/dataBroker.js, line 331

##### Returns:

A parsed and cleaned up JSON object where all of the CSV data is collated and organized and cleaned up ready for use.

Type
  object

#### (inner) processXmlData(data, contextName) → {object}

Does some final processing on JSON data loaded from an XML file, converting the data into a usable format and executes any additional data processing rules.

##### Parameters:

Name | Type | Description
--- | --- | ---
`data` | object | A JSON object that contains all of the data loaded from a XML file.
`contextName` | string | The name that should be used when adding the objects to the D data structure for data-sharing.

Author:
  - Seth Hollingsead
Source:
  - brokers/dataBroker.js, line 376

##### Returns:

A parsed and cleaned up JSON object where all of the XML data is collated and organized and cleaned up ready for use.

Type
  object

#### (inner) processXmlLeafNode(inputData, leafNodeName) → {object}

Recursively looks for the leaf node and restructures it from an array with a "$" child object to a single object entity with the name of the entity.

##### Parameters:

Name | Type | Description
--- | --- | ---
`inputData` | object | The data that should be recursively mutated to the correct data structure and returned.
`leafNodeName` | string | The leaf node name that we are looking for.

Author:
  - Seth Hollingsead
Source:
  - brokers/dataBroker.js, line 423

##### Returns:

The mutated object with the correct data structure.

Type
  object

#### (inner) scanDataPath(dataPath) → {array.<string>}

Scans the specified path and returns a collection of all the files contained recursively within that path and all sub-folders.

##### Parameters:

Name | Type | Description
--- | --- | ---
`dataPath` | string | The path that should be recursively scanned for files in all the folders.

Author:
  - Seth Hollingsead
Source:
  - brokers/dataBroker.js, line 31

##### Returns:

An array of strings that each have the full path and file name at all levels of the specified path including sub-folders.

Type
  array.<string>

#### (inner) setupDataStorage() → {void}

Does the initial setup of data storage on the D data structure.

Author:
  - Seth Hollingsead
Source:
  - brokers/dataBroker.js, line 546

##### Returns:

Nothing to return.

Type
  void

#### (inner) storeData(dataStorageContextName, dataToStore) → {boolean}

Stores some data in a data storage hive on the D data structure, under a caller specified sub-data storage hie name.

##### Parameters:

Name | Type | Description
--- | --- | ---
`dataStorageContextName` | string | The sub-data storage hive under-which the data should be stored.
`dataToStore` | string | boolean | integer | float | array | object | The data that should be stored, in any format.

Author:
  - Seth Hollingsead
Source:
  - brokers/dataBroker.js, line 560

##### Returns:

A True or False to indicate if the data storage was successful or not.

Type
  boolean

#### (inner) writeJsonDataToFile(fileToSaveTo, dataToWriteOut) → {boolean}

This is a wrapper function for businessRules.rules.fileOperations.writeJsonData.

##### Parameters:

Name | Type | Description
--- | --- | ---
`fileToSaveTo` | string | The full path to the file that should have the data written to it.
`dataToWriteOut` | object | The JSON data that should be written out to the specified JSON file.

Author:
  - Seth Hollingsead
Source:
  - brokers/dataBroker.js, line 525

##### Returns:

True or False to indicate if the file was saved successfully or not.

Type
  boolean