[![NodeJs](https://github.com/Belphemur/node-json-db/actions/workflows/nodejs.yml/badge.svg)](https://github.com/Belphemur/node-json-db/actions/workflows/nodejs.yml) [![codecov](https://codecov.io/gh/Belphemur/node-json-db/branch/master/graph/badge.svg?token=J3Ppt4UCbY)](https://codecov.io/gh/Belphemur/node-json-db) [![npm version](https://badge.fury.io/js/node-json-db.svg)](https://badge.fury.io/js/node-json-db) > A simple "database" that uses JSON file for Node.JS. ## Breaking changes since v1.x.x ### v2.0.0 JsonDB is now using the concept of async/await for all its calls since we read from the database file on demand and depending on how the database is configured, we might write at each push. - You're now forced to use the `Config` object to setup JsonDB - Every method is now asynchronous ## Installation Add `node-json-db` to your existing Node.js project. PNPM: ```bash pnpm add node-json-db ``` NPM: ```bash npm i node-json-db ``` ## Documentation [Auto Generated](https://belphemur.github.io/node-json-db) ## Known projects If you're looking for testing out the library in a playground [@Servant-Cities](https://github.com/Servant-Cities/) has made a small project to test it out: [Rest Api Playground](https://github.com/Servant-Cities/rest-api-playground-template) ## Inner Working ### Data The module stores the data using JavaScript Object directly into a JSON file. You can easily traverse the data to directly reach the desired property using the DataPath. The principle of DataPath is the same as XMLPath. ### Example ```javascript { test: { data1 : { array : ['test','array'] }, data2 : 5 } } ``` If you want to fetch the value of array, the DataPath is **/test/data1/array** To reach the value of data2 : **/test/data2** You can of course also get the full object **test** : **/test** Or even the root : **/** ## Usage See [test](https://github.com/Belphemur/node-json-db/tree/master/test) for more usage details. ```javascript import { JsonDB, Config } from "node-json-db"; // The first argument is the database filename. If no extension is used, '.json' is assumed and automatically added. // The second argument is used to tell the DB to save after each push // If you set the second argument to false, you'll have to call the save() method. // The third argument is used to ask JsonDB to save the database in a human readable format. (default false) // The fourth argument is the separator. By default it's slash (/) // The fifth argument enables sync writes (default false) var db = new JsonDB(new Config("myDataBase", true, false, "/", false)); // Pushing the data into the database // With the wanted DataPath // By default the push will override the old value await db.push("/test1", "super test"); // When pushing new data for a DataPath that doesn't exist, it automatically creates the hierarchy await db.push("/test2/my/test", 5); // You can also push objects directly await db.push("/test3", { test: "test", json: { test: ["test"] } }); // If you don't want to override the data but to merge them // The merge is recursive and works with Object and Array. await db.push( "/test3", { new: "cool", json: { important: 5, }, }, false, ); /* This give you this results : { "test":"test", "json":{ "test":[ "test" ], "important":5 }, "new":"cool" } */ // You can't merge primitives. // If you do this: await db.push("/test2/my/test/", 10, false); // The data will be overriden // Get the data from the root var data = await db.getData("/"); // Or from a particular DataPath var data = await db.getData("/test1"); // If you try to get some data from a DataPath that doesn't exist // You'll get an Error try { var data = await db.getData("/test1/test/dont/work"); } catch (error) { // The error will tell you where the DataPath stopped. In this case test1 // Since /test1/test does't exist. console.error(error); } // Easier than try catch when the path doesn't lead to data // This will return `myDefaultValue` if `/super/path` doesn't have data, otherwise it will return the data var data = (await db.getObjectDefault) < string > ("/super/path", "myDefaultValue"); // Deleting data await db.delete("/test1"); // Save the data (useful if you disable the saveOnPush) await db.save(); // In case you have an exterior change to the databse file and want to reload it // use this method await db.reload(); ``` ### TypeScript Support #### v0.8.0 As of v0.8.0, [TypeScript](https://www.typescriptlang.org) types are included in this package, so using `@types/node-json-db` is no longer required. #### v1.0.0 JsonDB isn't exported as default any more. You'll need to change how you load the library. This change is done to follow the right way to import module. ```javascript import { JsonDB, Config } from "node-json-db"; const db = new JsonDB(new Config("myDataBase", true, false, "/")); ``` #### Typing With TypeScript, you have access to a new method: getObject that will take care of typing your return object. ```typescript import { JsonDB, Config } from "node-json-db"; const db = new JsonDB(new Config("myDataBase", true, false, "/")); interface FooBar { Hello: string; World: number; } const object = { Hello: "World", World: 5 } as FooBar; await db.push("/test", object); // Will be typed as FooBar in your IDE const result = await db.getObject("/test"); ``` ### Array Support You can also access the information stored in arrays and manipulate them. ```typescript import { JsonDB, Config } from "node-json-db"; // The first argument is the database filename. If no extension is used, '.json' is assumed and automatically added. // The second argument is used to tell the DB to save after each push // If you set the second argument to false, you'll have to call the save() method. // The third argument is used to ask JsonDB to save the database in a human readable format. (default false) const db = new JsonDB(new Config("myDataBase", true, false, "/")); // This will create an array 'myarray' with the object '{obj:'test'}' at index 0 await db.push( "/arraytest/myarray[0]", { obj: "test", }, true, ); // You can retrieve a property of an object included in an array // testString = 'test'; var testString = await db.getData("/arraytest/myarray[0]/obj"); // Doing this will delete the object stored at the index 0 of the array. // Keep in mind this won't delete the array even if it's empty. await db.delete("/arraytest/myarray[0]"); ``` #### Appending in Array ```javascript // You can also easily append a new item to an existing array // This sets the next index with {obj: 'test'} await db.push( "/arraytest/myarray[]", { obj: "test", }, true, ); // The append feature can be used in conjuction with properties // This will set the next index as an object {myTest: 'test'} await db.push("/arraytest/myarray[]/myTest", "test", true); ``` #### Last Item in Array ```javascript // Add basic array await db.push("/arraytest/lastItemArray", [1, 2, 3], true); // You can easily get the last item of the array with the index -1 // This will return 3 await db.getData("/arraytest/lastItemArray[-1]"); // You can delete the last item of an array with -1 // This will remove the integer "3" from the array await db.delete("/arraytest/lastItemArray[-1]"); // This will return 2 since 3 just got removed await db.getData("/arraytest/lastItemArray[-1]"); ``` #### Count for Array ```javascript // await db.push("/arraytest/list", [{ id: 65464646155, name: "test" }], true); // You can request for the total number of elements, in this case, 1 let numberOfElements = await db.count("/arraytest/list"); ``` #### Get Index in Array ```javascript // You can have the current index of an object await db.push("/arraytest/myarray", [{ id: 65464646155, name: "test" }], true); await db.getIndex("/arraytest/myarray", 65464646155); // The default property is 'id' // You can add another property instead await db.getIndex("/arraytest/myarray", "test", "name"); // It's useful if you want to delete an object await db.delete( "/arraytest/myarray[" + (await db.getIndex("/arraytest/myarray", 65464646155)) + "]", ); ``` #### Nesting in Array ```javascript // You can easily access any nested array and their object // You can also append another array to a nested array await db.push( "/arraytest/myarray", [ [ { obj: "test", }, { obj: "hello", }, ], [ { obj: "world", }, ], ], true, ); // This will return the first object (obj: 'test') await db.getData("/arraytest/myarray[0][0]"); ``` #### Getting Item path in Array by id or Another key ```javascript await db.push( "/myarray", [ { id: "1", obj: "test", }, { id: "2", obj: "hello", }, { id: "3", obj: "hello", children: [ { id: "1", desc: "a sub item", }, ], }, ], true, ); // You can easily get the path of any nested array and its child object by a property using the route style syntax, the default is the object's "id" property const itemPath = db.fromPath("/myarray/3/children/1"); ``` #### Database Encryption #### Supported Encryption keys - Plain text string - Binary Buffer - Symmetric Key Object ( cf node crypto lib ) #### Usage ```typescript import { JsonDB, Config } from "node-json-db"; import { generateKeySync, randomBytes } from "crypto"; // First instanciate Config class const config = new Config("myDataBase", true, false, "/"); // create or retrieve binary cipher key of minimum 32 bytes const key = randomBytes(32); // create or retrieve string cipher key of minimum 32 bytes/chars // const key = "12345678901234567890123456789012" // create or retrieve symmetric key of minimum 32 bytes // const key = generateKeySync("hmac", { // length: 256, // }); // set encryption key into Config object // This will automatically change the database filename to use .enc.json extension // myDataBase.json becomes myDataBase.enc.json config.setEncryption(key); // instanciate database const db = new JsonDB(config); // use your database as you would normally do // Data will be encrypted and stored in myDataBase.enc.json await db.push("/test1", "super test"); ``` **Important Notes:** - When encryption is enabled, the database file extension automatically changes to `.enc.json` - This prevents accidentally accessing encrypted databases without the proper encryption key - Encrypted and non-encrypted databases use different file paths (e.g., `myDataBase.enc.json` vs `myDataBase.json`) - The encryption uses AES-256-GCM for secure authenticated encryption - The encryption key must be exactly 32 bytes #### Type Serialization JsonDB automatically serializes and deserializes JavaScript types that are not natively supported by JSON. The following types are supported out of the box: | Type | Serialized format | | -------- | ------------------------------------------------------------------------- | | `Date` | `{ "__type": "Date", "__value": "2023-01-01T00:00:00.000Z" }` | | `Set` | `{ "__type": "Set", "__value": [1, 2, 3] }` | | `Map` | `{ "__type": "Map", "__value": [["key", "value"]] }` | | `RegExp` | `{ "__type": "RegExp", "__value": { "source": "^test$", "flags": "i" } }` | | `BigInt` | `{ "__type": "BigInt", "__value": "9007199254740993" }` | ```typescript import { JsonDB, Config } from "node-json-db"; const db = new JsonDB(new Config("myDataBase")); // All these types are automatically serialized and deserialized await db.push("/data", { tags: new Set(["typescript", "json"]), metadata: new Map([ ["version", 1], ["author", "test"], ]), createdAt: new Date(), pattern: /^hello\s+world$/i, bigNumber: BigInt("9007199254740993"), }); const data = await db.getData("/data"); // data.tags is a Set, data.metadata is a Map, data.createdAt is a Date, etc. ``` #### Custom Serializers You can add support for additional types by implementing the `ISerializer` interface and registering them with `Config.addSerializer()`: ```typescript import { JsonDB, Config, ISerializer } from "node-json-db"; const urlSerializer: ISerializer = { type: "URL", serialize: (value: URL) => value.href, deserialize: (value: string) => new URL(value), test: (value: any) => value instanceof URL, }; const config = new Config("myDataBase"); config.addSerializer(urlSerializer); const db = new JsonDB(config); await db.push("/link", new URL("https://example.com")); const link = await db.getData("/link"); // URL instance ``` You can also use `defaultSerializers` directly with `JsonAdapter` for full control: ```typescript import { JsonAdapter, FileAdapter, defaultSerializers, ISerializer, } from "node-json-db"; const mySerializer: ISerializer = { /* ... */ }; const adapter = new JsonAdapter(new FileAdapter("mydb.json", false), false, [ ...defaultSerializers, mySerializer, ]); ``` ### Exception/Error #### Type | Type | Explanation | | ------------- | :-------------------------------------------------------------: | | DataError | When the error is linked to the Data Given | | DatabaseError | Linked to a problem with the loading or saving of the Database. | #### Errors | Error | Type | Explanation | | --------------------------------------------------------------------- | :-----------: | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | The Data Path can't be empty | DataError | The Database expects to receive at least the root **separator** as part of the DataPath. | | Can't find dataPath: /XXX. Stopped at YYY | DataError | When the full hierarchy of the provided DataPath is not present in the Database, it indicates the extent of its validity. This error can occur when using the `getData` and `delete` operations. | | Can't merge another type of data with an Array | DataError | This occurs if you chose not to override the data when pushing (merging) and the new data is an array, but the current data isn't an array (an Object for example). | | Can't merge an Array with an Object | DataError | Similar to the previous message, you have an array as the current data and request to merge it with an object. | | DataPath: /XXX. YYY is not an array. | DataError | When trying to access an object as an array. | | DataPath: /XXX. Can't find index INDEX in array YYY | DataError | When trying to access a non-existent index in the array. | | Only numerical values accepted for array index | DataError | An array can only use `number` for its indexes. For this use the normal object. | | The entry at the path (/XXX) needs to be either an Object or an Array | DataError | When using the `find` method, the rootPath needs to point to an object or an array to search within it for the desired value. | | Can't Load Database: XXXX | DatabaseError | JsonDB can't load the database for "err" reason. You can find the nested error in `error.inner` | | Can't save the database: XXX | DatabaseError | JsonDB can't save the database for "err" reason. You can find the nested error in `error.inner` | | DataBase not loaded. Can't write | DatabaseError | Since the database hasn't been loaded correctly, the module won't let you save the data to avoid erasing your database. | # Limitations ## Object with `separator` in key Object pushed with key containing the `separator` character won't be reachable. See [#75](https://github.com/Belphemur/node-json-db/issues/75). Please consider the `separator` as a reserved character by node-json-await db. # Contributors
Jamie Davis
Jamie Davis
πŸ›‘οΈ		 sidblommerswork
sidblommerswork
πŸ’» ⚠️		 Max Huber
Max Huber
πŸ’»		 Adam
Adam
πŸ’» πŸ“–		 Divine Anum
Divine Anum
πŸ“–		 MichaΕ‚ Fedyna
MichaΕ‚ Fedyna
πŸ“–		 alex.query
alex.query
πŸ’»
Tony Trupe
Tony Trupe
⚠️		 sh6986
sh6986
πŸ›		 pedroELT
pedroELT
πŸ’»		 stefnotch
stefnotch
πŸ’»
## Special Thanks [James Davis](https://github.com/davisjam) for helping to fix a regular expression vulnerable to [catastrophic backtracking](https://docs.microsoft.com/en-us/dotnet/standard/base-types/backtracking-in-regular-expressions). ## License [![FOSSA Status](https://app.fossa.io/api/projects/git%2Bgithub.com%2FBelphemur%2Fnode-json-await db.svg?type=large)](https://app.fossa.io/projects/git%2Bgithub.com%2FBelphemur%2Fnode-json-db?ref=badge_large)