DEV Community

Cover image for Just in! A New Persistent NoSQL Database (18 KiB only!)
Trinmar Boado
Trinmar Boado

Posted on

Just in! A New Persistent NoSQL Database (18 KiB only!)

#javascript #webdev #database #sql

Welcome to Trin.DB!

A fast RESTful persistent or in memory NoSQL database (18 KiB only!)

Github Repo: https://github.com/trinly01/TrinDB

Installation 

npm install trin.db
Enter fullscreen mode Exit fullscreen mode

or

yarn add trin.db
Enter fullscreen mode Exit fullscreen mode

Usage 

const express = require('express')
const app = express()
const port = process.env.PORT || 3000
const trinDB = require('trin.db')

app.use(express.json())               // required for RESTful APIs

app.listen(port, async () => {
  app.trinDB = {
    todos: await trinDB({             // Todos Service
      filename: 'trinDb/todos.db',    // get records from a file
      inMemoryOnly: false,            // Optional
      restful                         // Optional
    })
  }
})

// Other Options

const restful = {                     // Optional
  app,                                // express app
  url: '/todos',                      // API end-point
  hooks                               // Optional
}

const hooks = ({app, service}) => ({  // Hooks Example
  before: {
    all: [
      (req, res, next) => {
        console.log('before all hook')
        next()
      }
    ],
    get: [],
    find: [],
    create: [],
    patch: [],
    remove: []
  }
  after: {
    all: [
      (result) => {
        console.log(result)
        return result
      }
    ],
  }
})
Enter fullscreen mode Exit fullscreen mode

await trinDB(<object>)

Returns a trinDB Service

Object Prop Type Description Default Mode
filename <string> Path to file. Required if in persistent mode n/a persistent
inMemoryOnly <boolean> ( Optional ) If true, database will be in non-persistent mode false in-memory
restful <object> ( Optional ) { app, url, hooks } n/a persistent

service.create(<object>)

Returns the created <object>

/* example existing records (service.data)
  {
    asd: { _id: 'asd', text: 'Hello', read: true, nested: { prop: 'xander' } },
    zxc: { _id: 'zxc', text: 'World', read: false, nested: { prop: 'ford' } }
  }
*/

const result = service.create({
  text: 'Trinmar Pogi'
})

console.log(result)
// { _id: 'qwe', text: 'Trinmar Pogi' }

console.log(service.data)
/* service.data with the newly created object
  {
    asd: { _id: 'asd', text: 'Hello', read: true, nested: { prop: 'xander' } },
    zxc: { _id: 'zxc', text: 'World', read: false, nested: { prop: 'ford' } },
    qwe: { _id: 'qwe', text: 'Trinmar Pogi' }
  }
*/
Enter fullscreen mode Exit fullscreen mode

RESTful API

curl --location --request POST 'http://localhost:3000/todos' \
--header 'Content-Type: application/json' \
--data-raw '{
  "text": "Trinmar Pogi"
}'
Enter fullscreen mode Exit fullscreen mode

service.find(<object>)

Returns found data <object>

/* example existing records (service.data)
  {
    asd: { _id: 'asd', firstName: 'Trinmar', lastName: 'Pogi', age: 20 },
    zxc: { _id: 'zxc', firstName: 'Trinly Zion', lastName: 'Boado', age: 1 },
    qwe: { _id: 'qwe', firstName: 'Lovely', lastName: 'Boado', age: 18 }
  }
*/

// Equality
result = service.find({
  query: {
    lastName: 'Pogi' // equality
  },
  limit: 10, // default 10
  skip: 0 // default 0
})
console.log(result)
/*
  {
    total: 1,
    limit: 10,
    skip: 0,
    data: {
      asd: { _id: 'asd', firstName: 'Trinmar', lastName: 'Pogi', age: 20 }
    }
  }
*/
Enter fullscreen mode Exit fullscreen mode

RESTful API

curl --location --request GET 'http://localhost:3000/todos?lastName=Pogi&$limit=10&$skip=0'
Enter fullscreen mode Exit fullscreen mode

Complex Query (conditional >, >==, <, <==, &&, || ) 

// Map data or select specific props
result = service.find({
  query (obj) {
    return ob.age < 20
  },
  map (obj) {
    return {
      fullName: obj.firstName + ' '+  obj.lastName
    }
  }
})
console.log(result)
/*
  {
    total: 2,
    limit: 10,
    skip: 0,
    data: {
      zxc: { _id: 'zxc', firstName: 'Trinly Zion Boado' },
      qwe: { _id: 'qwe', firstName: 'Lovely Boado' }
    }
  }
*/
Enter fullscreen mode Exit fullscreen mode

service.search(keywords)

fuzzy search finds data based on the keywords (<String>) and returns it sorted by _score

/* example existing records (service.data)
  {
    asd: { _id: 'asd', firstName: 'Trinmar', lastName: 'Boado' },
    zxc: { _id: 'zxc', firstName: 'Trinly Zion', lastName: 'Boado' },
    qwe: { _id: 'qwe', firstName: 'Lovely', lastName: 'Boado' }
  }
*/

result = service.search('ly oad')

console.log(result)
/*
  {
    total: 3,
    data: {
      qwe: { _score: 2, _id: 'qwe', firstName: 'Lovely', lastName: 'Boado', age: 18 },
      zxc: { _score: 2, _id: 'zxc', firstName: 'Trinly Zion', lastName: 'Boado', age: 1 },
      asd: { _score: 1, _id: 'asd', firstName: 'Trinmar', lastName: 'Pogi', age: 20 },
    }
  }
*/
Enter fullscreen mode Exit fullscreen mode

RESTful API

curl --location --request GET 'http://localhost:3000/todos?$search=ly%20oad'
Enter fullscreen mode Exit fullscreen mode

service.patch(_id, <object>)

Returns the created <object>

// { _id: 'q12m3k', firstName: 'Trinmar', lastName: 'Boado' nested: { counter: 123 } }

const result = service.patch('q12m3k', {
  lastName: 'Pogi',
  children: ['Trinly Zion'],
  'nested.counter': 456
})

console.log(result)
// { _id: 'q12m3k', lastName: 'Pogi' children: ['Trinly Zion'], 'nested.counter': 456 }

console.log(service.data['q12m3k'])
// { _id: 'q12m3k', firstName: 'Trinmar', lastName: 'Pogi', nested: { prop: 456 }, children: ['Trinly Zion'] }
Enter fullscreen mode Exit fullscreen mode

RESTful API

curl --location --request PATCH 'http://localhost:3000/todos/:_id' \
--header 'Content-Type: application/json' \
--data-raw '{
    "lastName": "Pogi",
    "children": ["Trinly Zion"],
    "nested.counter": 456
}'
Enter fullscreen mode Exit fullscreen mode

service.remove(_id)

Returns the removed <object>

service.remove('q12m3k')

console.log(service.data['q12m3k'])
// undefined
Enter fullscreen mode Exit fullscreen mode

RESTful API

curl --location --request DELETE 'http://localhost:3000/todos/:_id'
Enter fullscreen mode Exit fullscreen mode

service.removeProps(_id, <object>)

Returns the removed <object> props

// { _id: 'q12m3k', firstName: 'Trinmar', lastName: 'Pogi', nested: { prop: 456 }, children: ['Trinly Zion'] }

service.removeProps('q12m3k', {
  lastName: true,
  'nested.prop': true
  firstName: false
})

console.log(service.data['q12m3k'])
// { _id: 'q12m3k', firstName: 'Trinmar', children: ['Trinly Zion'] }
Enter fullscreen mode Exit fullscreen mode

RESTful API

curl --location --request PATCH 'http://localhost:3000/todos/:_id' \
--header 'Content-Type: application/json' \
--data-raw '{
    "$action": "removeProps"
    "lastName": true,
    "nested.prop": true,
    "firstName": false
}'
Enter fullscreen mode Exit fullscreen mode

service.inc(_id, <object>)

Increments specific props and returns the <object>

// { _id: 'q12m3k', firstName: 'Trinmar', lastName: 'Pogi', nested: { prop: 456 }, children: ['Trinly Zion'] }

service.inc('q12m3k', {
  'nested.prop': 5
})

console.log(service.data['q12m3k'])
// { _id: 'q12m3k', firstName: 'Trinmar', lastName: 'Pogi', nested: { prop: 461 }, children: ['Trinly Zion'] }
Enter fullscreen mode Exit fullscreen mode

RESTful API

curl --location --request PATCH 'http://localhost:3000/todos/:_id' \
--header 'Content-Type: application/json' \
--data-raw '{
    "$action": "inc"
    "nested.prop": 5
}'
Enter fullscreen mode Exit fullscreen mode

service.splice(_id, <object>)

removes element by index and returns the <object>

// { _id: 'q12m3k', children: ['Trinly Zion', 'Trinmar Boado'] }

service.splice('q12m3k', {
  'children': 1
})

console.log(service.data['q12m3k'])
// { _id: 'q12m3k', children: ['Trinly Zion'] }
Enter fullscreen mode Exit fullscreen mode

RESTful API

curl --location --request PATCH 'http://localhost:3000/todos/:_id' \
--header 'Content-Type: application/json' \
--data-raw '{
    "$action": "splice"
    "children": 1
}'
Enter fullscreen mode Exit fullscreen mode

service.push(_id, <object>)

adds one or more elements to the end of an array and returns the <object>

// { _id: 'q12m3k', children: ['Trinly Zion', 'Trinmar Boado'] }

service.push('q12m3k', {
  'children': 'Lovely Boado'
})

console.log(service.data['q12m3k'])
// { _id: 'q12m3k', children: ['Trinly Zion', 'Trinmar Boado', 'Lovely Boado'] }
Enter fullscreen mode Exit fullscreen mode

RESTful API

curl --location --request PATCH 'http://localhost:3000/todos/:_id' \
--header 'Content-Type: application/json' \
--data-raw '{
    "$action": "push"
    "children": "Lovely Boado'"
}'
Enter fullscreen mode Exit fullscreen mode

service.unshift(_id, <object>)

adds one or more elements to the beginning of an array and returns the <object>

// { _id: 'q12m3k', children: ['Trinly Zion', 'Trinmar Boado'] }

service.unshift('q12m3k', {
  'children': 'Lovely Boado'
})

console.log(service.data['q12m3k'])
// { _id: 'q12m3k', children: ['Lovely Boado', 'Trinly Zion', 'Trinmar Boado'] }
Enter fullscreen mode Exit fullscreen mode

RESTful API

curl --location --request PATCH 'http://localhost:3000/todos/:_id' \
--header 'Content-Type: application/json' \
--data-raw '{
    "$action": "unshift"
    "children": "Lovely Boado'"
}'
Enter fullscreen mode Exit fullscreen mode

service.sort(data,<object>)

Sorts the data based on the <object> and returns the sorted data

/* example existing records (service.data)
  {
    asd: { _id: 'asd', firstName: 'Trinmar', lastName: 'Pogi', age: 20 },
    zxc: { _id: 'zxc', firstName: 'Trinly Zion', lastName: 'Boado', age: 1 },
    qwe: { _id: 'qwe', firstName: 'Lovely', lastName: 'Boado', age: 18 }
  }
*/

// Descending (-1)
result = service.sort({
  data: service.data, // (Optional) if not defined, service.data will be used
  params: {
    age: -1
  }
})

console.log(result)
/*
  {
    asd: { _id: 'asd', firstName: 'Trinmar', lastName: 'Pogi', age: 20 },
    qwe: { _id: 'qwe', firstName: 'Lovely', lastName: 'Boado', age: 18 },
    zxc: { _id: 'zxc', firstName: 'Trinly Zion', lastName: 'Boado', age: 1 }
  }
*/
Enter fullscreen mode Exit fullscreen mode

service.copmact(filename, <object>)

writes the compact data to a file
| param | Type | Description | Default |
|--|--|--|--|
| filename | <string> | (Optional) Path to file | current |
| | <object> | ( Optional ) a TrinDB object | service.data |

service.copmact('test.db', service.data)
Enter fullscreen mode Exit fullscreen mode

Github Repo: https://github.com/trinly01/TrinDB

Join and support our Community
Web and Mobile Developers PH
[ Facebook Page | Group ]

Top comments (12)

Collapse
 
thecodrr profile image
Abdullah Atta •

Would love if you showed some benchmarks instead of copy-pasting the whole API here.

Collapse
 
trinly01 profile image
Trinmar Boado •

Hi Atta,
Nice suggestion!
You may contribute to this project by adding a benchmark.
Will love to merge it ❤️

Collapse
 
thecodrr profile image
Abdullah Atta •

No it's your job to provide benchmarks. :(

Thread Thread
 
rezanop profile image
Rezanop •

To be honest, I get your point, but I don't like this wording when it comes to open source projects :)

Thread Thread
 
thecodrr profile image
Abdullah Atta •

Projects are projects. You can't expect other people to provide benchmarks for your project. You are the one advertising it, so you need to provide benchmarks. So yes, it is your job. Open source is not an excuse.

Collapse
 
trinly01 profile image
Trinmar Boado •

npmjs.com/package/msgpack

node-msgpack is currently slower than the built-in JSON.stringify() and JSON.parse() methods. In recent versions of node.js, the JSON functions have been heavily optimized. node-msgpack is still more compact, and we are currently working performance improvements. Testing shows that, over 500k iterations, msgpack.pack() is about 5x slower than JSON.stringify(), and msgpack.unpack() is about 3.5x slower than JSON.parse().

Collapse
 
roelofjanelsinga profile image
Roelof Jan Elsinga •

Looks very interesting! How is the performance when you have 100.000 - 1.000.000 records?

Collapse
 
trinly01 profile image
Trinmar Boado •

Hi Roelof, Contributors are welcome.
Need help to test it using real world scenarios.

Collapse
 
patarapolw profile image
Pacharapol Withayasakpunt •

Currently I am interested in JSON schema validation, perhaps with Ajv.

Collapse
 
trinly01 profile image
Trinmar Boado • • Edited

Thanks for the suggestion. <3
I already converted it from appendFileSync to fileWriteStream

Collapse
 
trinly01 profile image
Trinmar Boado •

All suggestions are being considered. Some are already implemented. Thank you to all of insights and inputs ❤️

 
trinly01 profile image
Trinmar Boado •

Thanks for the detailed explanation ❤️

My goal is to be pure JS implementation with no binary dependencies.

I think streams are optimized to minimize CPU and memory overheads

View full discussion (12 comments)

Read next

mountainhokie profile image

Moving beyond console.log

Justin -

grzegorz_dubiel_db99203fe profile image

Async Local Storage is Here to Help You

Grzegorz Dubiel -

chloewilliams profile image

Introducing Milvus 2.5: Built-in Full-Text Search, Advanced Query Optimization, and More 🚀

Chloe Williams -

rimi_jalil2002 profile image

Can English Replace Java? The Future of Programming in Plain Language

Rimsha Jalil -