Indicates the orientation of the data within the file.
Most CSV data files are 'values' (row-oriented).
JSON files can commonly be either. Records are probably more common, though require more space due to replication of keys. Apache Arrow or Parquet are columnar. This nearly aligns with pandas: https://pandas.pydata.org/pandas-docs/stable/user\_guide/io.html\#json
A key difference (which probably needs resolved) is that we don't yet support the notion of an index. See their example for "columns" or "index" orientation, which is a nested structure.
Example JSON formats: records: \[{ colA: valueA1, colB: valueB1 }, { colA: valueA2, colB: valueB2 }\]
columnar: { colA: \[valueA1, valueA2\], colB: \[valueB1, valueB2\] }
array: \["value1", "value2"\]
values: \[ \["colA", "colB"\], \["valueA1", "valueA2"\], \["valueA2", "valueB2"\] \]
| | [DataType](./schema.datatype.md) | Explicit data type of the value (i.e., for a column or property). TODO: clarify/update null/undefined | | [DateComparisonOperator](./schema.datecomparisonoperator.md) | | | [ErrorCode](./schema.errorcode.md) | | | [FieldAggregateOperation](./schema.fieldaggregateoperation.md) | This is the subset of aggregate functions that can operate on a single field so we don't accommodate additional args. See https://uwdata.github.io/arquero/api/op\#aggregate-functions | | [FilterCompareType](./schema.filtercomparetype.md) | Indicates the comparison type used for a filter operation. This is done on a row-by-row basis. | | [JoinStrategy](./schema.joinstrategy.md) | | | [KnownProfile](./schema.knownprofile.md) | | | [KnownRel](./schema.knownrel.md) | | | [MathOperator](./schema.mathoperator.md) | | | [MergeStrategy](./schema.mergestrategy.md) | | | [NumericComparisonOperator](./schema.numericcomparisonoperator.md) | | | [ParseType](./schema.parsetype.md) | This is a subset of data types available for parsing operations. | | [SetOp](./schema.setop.md) | Indicates the type of set operation to perform across two collections. | | [SortDirection](./schema.sortdirection.md) | | | [StringComparisonOperator](./schema.stringcomparisonoperator.md) | | | [VariableNature](./schema.variablenature.md) | Describes the semantic shape of a variable. This has particular effect on how we display and compare data, such as using line charts for continuous versus bar charts for categorical. This mostly applies to numeric variables, but strings for instance can be categorical. | | [Verb](./schema.verb.md) | | | [WindowFunction](./schema.windowfunction.md) | These are operations that perform windowed compute. See https://uwdata.github.io/arquero/api/op\#window-functions | ## Functions | Function | Description | | --- | --- | | [createCodebookSchemaObject(input)](./schema.createcodebookschemaobject.md) | | | [createDataPackageSchemaObject(input)](./schema.createdatapackageschemaobject.md) | | | [createDataTableSchemaObject(input)](./schema.createdatatableschemaobject.md) | | | [createSchemaValidator()](./schema.createschemavalidator.md) | | | [createTableBundleSchemaObject(input)](./schema.createtablebundleschemaobject.md) | | | [createWorkflowSchemaObject(input)](./schema.createworkflowschemaobject.md) | | ## Interfaces | Interface | Description | | --- | --- | | [AggregateArgs](./schema.aggregateargs.md) | | | [BasicInput](./schema.basicinput.md) | Single-input, single-output step I/O | | [Bin](./schema.bin.md) | Describes a data bin in terms of inclusive lower bound and count of values in the bin. | | [BinArgs](./schema.binargs.md) | | | [BinarizeArgs](./schema.binarizeargs.md) | | | [BooleanArgs](./schema.booleanargs.md) | | | [BundleSchema](./schema.bundleschema.md) | A schema for defining custom bundle types. | | [Category](./schema.category.md) | Describes a nominal category in terms of category name and count of values in the category. | | [CodebookSchema](./schema.codebookschema.md) | This contains all of the field-level details for interpreting a dataset, including data types, mapping, and metadata. Note that with persisted metadata and field examples, a dataset can often be visualized and described to the user without actually loading the source file. resource profile: 'codebook' | | [Constraints](./schema.constraints.md) | Validation constraints for a field. | | [ConvertArgs](./schema.convertargs.md) | | | [CopyArgs](./schema.copyargs.md) | | | [Criterion](./schema.criterion.md) | | | [DataPackageSchema](./schema.datapackageschema.md) | Defines a Data Package, which is a collection of data resources such as files and schemas. Loosely based on the Frictionless spec, but modified where needed to meet our needs. https://specs.frictionlessdata.io/data-package/ | | [DataShape](./schema.datashape.md) | Defines parameters for understanding the logical structure of data contents. | | [DataTableSchema](./schema.datatableschema.md) | This defines the table-containing resource type. A dataset can be embedded directly using thedata property, or it can be linked to a raw file using the path. If the latter, optional format and parsing options can be applied to aid interpreting the file contents. resource profile: 'datatable' |
| [DeriveArgs](./schema.deriveargs.md) | |
| [DestructureArgs](./schema.destructureargs.md) | |
| [DualInput](./schema.dualinput.md) | Dual-input, single-output step I/O |
| [EncodeDecodeArgs](./schema.encodedecodeargs.md) | |
| [EraseArgs](./schema.eraseargs.md) | |
| [Field](./schema.field.md) | Contains the full schema definition and metadata for a data field (usually a table column). This includes the required data type, various data nature and rendering properties, potential validation rules, and mappings from a data dictionary. |
| [FieldError](./schema.fielderror.md) | |
| [FieldMetadata](./schema.fieldmetadata.md) | Holds core metadata/stats for a data field. |
| [FillArgs](./schema.fillargs.md) | |
| [FilterArgs](./schema.filterargs.md) | |
| [FoldArgs](./schema.foldargs.md) | |
| [ImputeArgs](./schema.imputeargs.md) | |
| [InputColumnArgs](./schema.inputcolumnargs.md) | |
| [InputColumnListArgs](./schema.inputcolumnlistargs.md) | Base interface for a number of operations that work on a column list. |
| [InputColumnRecordArgs](./schema.inputcolumnrecordargs.md) | |
| [InputKeyValueArgs](./schema.inputkeyvalueargs.md) | |
| [JoinArgs](./schema.joinargs.md) | |
| [JoinArgsBase](./schema.joinargsbase.md) | |
| [LookupArgs](./schema.lookupargs.md) | |
| [MergeArgs](./schema.mergeargs.md) | |
| [Named](./schema.named.md) | Base interface for sharing properties of named resources/objects. |
| [OnehotArgs](./schema.onehotargs.md) | |
| [OrderbyArgs](./schema.orderbyargs.md) | |
| [OrderbyInstruction](./schema.orderbyinstruction.md) | |
| [OutputColumnArgs](./schema.outputcolumnargs.md) | |
| [ParserOptions](./schema.parseroptions.md) | Parsing options for delimited files. This is a mix of the options from pandas and spark. |
| [PivotArgs](./schema.pivotargs.md) | |
| [PrintArgs](./schema.printargs.md) | |
| [RecodeArgs](./schema.recodeargs.md) | |
| [RelationshipConstraint](./schema.relationshipconstraint.md) | |
| [ResourceSchema](./schema.resourceschema.md) | Parent class for any resource type understood by the system. Any object type that extends from Resource is expected to have a standalone schema published. For project state, this can be left as generic as possible for now. |
| [RollupArgs](./schema.rollupargs.md) | |
| [SampleArgs](./schema.sampleargs.md) | |
| [SpreadArgs](./schema.spreadargs.md) | |
| [StepJsonCommon](./schema.stepjsoncommon.md) | Common step properties |
| [StringsArgs](./schema.stringsargs.md) | |
| [StringsReplaceArgs](./schema.stringsreplaceargs.md) | |
| [TableBundleSchema](./schema.tablebundleschema.md) | A table bundle encapsulates table-specific resources into a single resource with a prescribed workflow.
A tablebundle requires a source entry with rel="input" for the source table. A tablebundle may also include source entries with rel="codebook" and rel="workflow" for interpretation and processing of the source data table.