Jan Küster

Posted on Aug 7, 2022 • Edited on Feb 16, 2023

Meteor toolbox for your next hackathon or project start

#javascript #meteor #backend

Whether you start a new project or want to get ready for the next hackathon: These tools will help you to get fast results and keep code maintainable and scalable at the same time. Teammates will better understand what your code does by reading less, but more descriptive code.

At lea.online we often have created similar implementations across apps when managing collections, methods, publications, file uploads and http routes. We also decided to abstract some of these recurring pattern and published them as Meteor packages under a free license so you can benefit from our efforts, too. 💪

Code splitting

Meteor supports to write code once and use it on the server AND on the client. It also supports exact code-splitting at build-time, which allows you to indicate, if code is only meant for server or client. Both features together allow for isomorphic code, which means that an object "looks" similar (same functions/api) but can be implemented to behave diferrent, specifically for each environment.

In such a case you don't want code to leak into the other environment, which could have negative side-effects like increased client bundle size (thus longer load times). To achieve that and keep the code tidy and readable, we have created some short and easy functions:

// returns a value only on a given architecture
export const onServer = x => Meteor.isServer ? x : undefined
export const onClient = x => Meteor.isClient ? x : undefined

// execute a function and return it's value only on a given architecture
export const onServerExec = fn => Meteor.isServer ? fn() : undefined
export const onClientExec = fn => Meteor.isClient ? fn() : undefined

An example of code-splitting with direct assignment:

import { onServer, onClient } from 'utils/arch'

export const Greeting = {
  name: onServer('John Doe'), // will be undefined on the client
  age: onClient(42) // will be undefined on the server 
}

Note, that this example is note isomorphic. On the server name will be present but not on the client. Vice versa with age.

An example of isomorphic code using code-splitting with function execution would be:

import { onServerExec, onClientExec } from 'utils/arch'

export const Greeting = {}

Greeting.name = undefined

onServerExec(() => {
  Greeting.name = 'John Doe'
})

onClientExec(() => {
  Greeting.name = 'Anonymous'
})

The great thing is, that you can use import within these function calls and still prevent leakage into the other environment!

Bonus: create an isomorphic wrapper

The above abstractions can be taken even further to wrap assignments into one line:

export const isomorphic = ({ server, client }) => {
  if (Meteor.isServer) return server
  if (Meteor.isClient) return client
}

export const isomorphicExec = ({ server, client }) => {
  if (Meteor.isServer) return server()
  if (Meteor.isClient) return client()
}

Usage:

export const Greeting = {
  name: isomorphic({
    server: 'John Doe',
    client: 'Anonymous'
  }),

  lang: isomorphicExec({
    server: () => Meteor.settings.defaultLang,
    client: () => window.navigator.language
  })
}

When to use which one?

Use the direct assignments if values are static or have no external dependencies or if you want to create non-isomorphic objects. Otherwise use the function-based assignments to prevent leaking dependencies into the other environment.

Collection factory

GitHub Link: https://github.com/leaonline/collection-factory
Packosphere: https://packosphere.com/leaonline/collection-factory

Creating Mongo collections in Meteor is already as fast and easy as it ever can be. 🚀 Beyond that you may also want to attach a schema to a given collection, attach hooks or define the collection as local.

In order to achieve all this at once you can leverage the power of our lea.online collection factory package. Add the following packages to your project:

$ meteor add leaonline:collection-factory aldeed:collection2
$ meteor npm install --save simpl-schema

The other two packages (simpl-schema and aldeed:collection2) allow to validate any incoming data on the collection level (insert/update/remove), providing an extra layer of safety when it comes to handling your data. Note, that they are entirely optional.

Use-case

Let's consider a collection definition as an Object with a name (representing the collection name) and schema (representing the collection data structure schema):

export const Todos = {
  name: 'todos'
}

Todos.schema = {
  userId: String,
  title: String,
  isPublic: Boolean,
  items: Array,
  'items.$': Object,
  'items.$.text': String,
  'items.$.checked': Boolean
}

As you can see there is no code for actually creating (instantiating) the collection and neither for creating a new schema. But it looks readable and clear to understand. This is where the factory comes into play:

// imports/api/factories/createCollection
import { createCollectionFactory } from 'meteor/leaonline:collection-factory'
import SimpleSchema from 'simpl-schema'

export const createCollection = createCollectionFactory({ 
  schemaFactory: definitions => new SimpleSchema(definitions) 
})

With this function you have now a single handler to create collections and attach a schema to them:

// server/main.js
import { Todos } from '../imports/api/Todos'
import { createCollection } from '/path/to/createCollection'

const TodosCollection = createCollection(Todos)
TodosCollection.insert({ foo: 'bar' }) // throws validation error, foo is not in schema

Collection instances

Separating definition from implementation is what makes your code testable and maintainable. You can even go one step further and access these collections independently (decoupled) from the local collection variable.

In order to do that you need to add the Mongo Collection Instances package (dburles:mongo-collection-instances):

Meteor-Community-Packages / mongo-collection-instances

🗂 Meteor package allowing Mongo Collection instance lookup by collection name

Meteor Mongo Collection Instances

This package augments Mongo.Collection (and the deprecated Meteor.Collection) and allows you to later lookup a Mongo Collection instance by the collection name.

Installation

$ meteor add dburles:mongo-collection-instances

Usage Example

const Books = new Mongo.Collection('books');

Mongo.Collection.get('books').insert({ name: 'test' });

Mongo.Collection.get('books').findOne({ name: 'test' });

API

Mongo.Collection.get('name', [options])

Returns the collection instance.

name (String)
options (Object) [optional]
- options.connection (A connection object, see example below)

Mongo.Collection.getAll()

Returns an array of objects containing:

name (The name of the collection)
instance (The collection instance)
options (Any options that were passed in on instantiation)

Multiple connections

It's possible to have more than one collection with the same name if they're on a different connection In order to lookup the…

View on GitHub

$ meteor add dburles:mongo-collection-instances

Wrapper:

export const getCollection = ({ name }) => Mongo.Collection.get(name)

Usage:

const TodosCollection = getCollection(Todos)

Now you can access collections anywhere. We will use this pattern in later sections to use our defined collections anywhere in the code executions.

Method Factory

GitHub Link: https://github.com/leaonline/method-factory
Packosphere: https://packosphere.com/leaonline/method-factory

A central concept of Meteor is to define rpc-style endpoints, named Meteor Methods. They can be called by any connected clients, which makes them a handy way to communicate with the server but also an easy attack vector. Many concepts around methods exist to validate incoming data or restrict access.

We published leaonline:method-factory as a way to easily define methods in a mostly declarative way. It does so by extending the concept of mdg:validated-method by a few abstractions:

import { createMethodFactory } from 'meteor/leaonline:method-factory'
import SimpleSchema from 'simpl-schema'

export const createMethod = createMethodFactory({
  schemaFactory: definitions => new SimpleSchema(definitions) 
})

If you don't have Simple Schema installed, you need to add it via npm:

$ meteor npm install --save simpl-schema

Defining a method for our Todos is super easy:

Todos.methods = {
  create: {
    name: 'todos.methods.create',
    schema: Todos.schema, // no need to write validate function
    isPublic: false, // see mixins section
    run: onServer(function (document) {
      document.userId = this.userId // don't let clients decide who owns a document
      return getCollection(Todos.name).insert(document)
    })
  }
}

Creating the Method is similar to the previously mentioned collection factory:

import { Todos } from '/path/to/Todos'
import { createMethod } from '/path/to/createMethod'

Object.methods(Todos).forEach(options => createMethod(options))

And finally, calling it on the client is also super easy:

import { Todos } from '/path/to/Todos'

const insertDoc = {
  title: 'buy groceries',
  isPublic: false,
  items: [
    { text: 'bread', checked: false },
    { text: 'butter', checked: false },
    { text: 'water', checked: false },
  ]
}

Meteor.call(Todos.methods.create.name, insertDoc)

Adding constraints via mixins

Let's say you want to make Todos private and let only their owners create/read/update/delete them. At the same time you want to log any errors that occur during a method call or due to permission being denied. You can use mixins - functions that extend the execution of a method, to get to these goals:

export const checkPermissions = options => {
  const { run, isPublic } = options

  // check if we have an authenticated user
  options.run = function (...args) {
    const env = this

    // methods, flagged as isPublic have no permissions check
    if (!isPublic && !env.userId) {
      throw new Meteor.Error('permissionDenied', 'notLoggedIn')
    }

    // if all good run the original function
    return run.apply(env, args)
  }

  return options
}

// note: replace the console. calls with
// your custom logging library

export const logging = options => {
   const { name, run } = options
   const logname = `[${name}]:`

   options.run = function (...args) {
     const env = this
     console.log(logname, 'run by', env.userId)

     try {
       run.apply(env, args)
     } catch (runtimeError) {
       console.error(logname, 'error at runtime')
       throw runtimeError
     }
   }

   return options
}

This is how the updated method factory looks like using mixins:

import { createMethodFactory } from 'meteor/leaonline:method-factory'
import SimpleSchema from 'simpl-schema'
import { checkPermissions } from '/path/to/checkPermissions'
import { logging } from '/path/to/loggin'

export const createMethod = createMethodFactory({
  schemaFactory: definitions => new SimpleSchema(definitions),
  mixins: [checkPermissions, logging]
})

These mixins are now applied to all methods automatically, without the need to assign them to each method on your own! Note, that the package would also allow to attach mixins only to a single method definition. If you want to read on the whole API documentation, you should checkout the repo:

leaonline / method-factory

Create validated Meteor methods. Lightweight. Simple.

Meteor ValidatedMethod Factory

Create validated Meteor methods. Lightweight. Simple.

With this package you can define factory functions to create a variety of Meteor methods Decouples definition from instantiation (also for the schema) and allows different configurations for different types of methods.

Minified size < 2KB!

Why do I want this?

Decouple definition from instantiation
Just pass in the schema as plain object, instead of manually instantiating SimpleSchema
Create fixed mixins on the abstract factory level, on the factory level, or both (see mixins section)

Installation

Simply add this package to your meteor packages

$ meteor add leaonline:method-factory

Usage

Import the createMethodFactory method and create the factory function from it:

import { createMethodFactory } from 'meteor/leaonline:method-factory'
const createMethod = createMethodFactory() // no params = use defaults
const fancyMethod = createMethod({ name: 'fancy', validate: () => {}, run: () =>

…

View on GitHub

Publication Factory

GitHub Link: https://github.com/leaonline/publication-factory
Packosphere: https://packosphere.com/leaonline/publication-factory

In Meteor you can subscribe to live updates of your Mongo collections. Meteor then takes care of all the syncing between server and the client. The server, however has to publish the data with all constraints as with methods (input validation, permissions etc.).

We created a handy abstraction for publications in order to have a similar API like the method factory (or like ValidatedMethod). It also allows you to pass mixins as with methods and this even reuse the mixins! Let's create our publication factory:

import { createPublicationFactory } from 'meteor/leaonline:publication-factory'
import { checkPermissions } from '/path/to/checkPermissions'
import { logging } from '/path/to/loggin'

const createPublication = createPublicationFactory({
  schemaFactory: definitions => new SimpleSchema(definitions),
  mixins: [checkPermissions, logging]
})

Then add a publication to our Todos that publishes a single todos list:

Todos.publications = {
  my: {
    name: 'todos.publications.my',
    schema: {
      limit: {
        type: Number,
        optional: true
        min: 1
      }
    },
    run: onServer(function ({ limit = 15 } = {}) {
      const query = { userId: this.userId }
      const projection = { limit }
      return getCollection(Todos).find(query, projection)
    })
  }
}

The creational pipeline is the same as the one we use for our methods:

import { Todos } from '/path/to/Todos'
import { createPublication } from '/path/to/createPublication'

Object.values(Todos.publications).forEach(options => createPublication(options))

There are multiple benefits at this point:

You have a readable and (mostly) descriptive way of defining, what Todos actually publishes.
You can reuse the mixins you used in the Methods Factory.
You can easily compose these factory methods together (which we will do in the last section)

For more info on the package you should read the documentation on the GitHub repository:

leaonline / publication-factory

Create Meteor publications. Lightweight. Simple.

Meteor Publication Factory

Create validated Meteor publications. Lightweight. Simple.

With this package you can define factory functions to create a variety of Meteor publications Decouples definition from instantiation (also for the schema) and allows different configurations for different types of publications.

Minified size < 2KB!

Why do I want this?

Decouple definition from instantiation
Validate publication arguments as with mdg:validated-method
Just pass in the schema as plain object, instead of manually instantiating SimpleSchema
Create mixins (similar to mdg:validated-method) on the abstract factory level, on the factory level, or both (see mixins section)
Fail silently in case of errors (uses the publication's error and ready), undefined cursors or unexpected returntypes

Installation

Simply add this package to your meteor packages

$ meteor add leaonline:publication-factory

Usage

Import the createPublicationFactory publication and create the factory function from it:

import { createPublicationFactory } from 'meteor/leaonline:publication-factory'
import { MyCollection } from '/path/to/MyCollection'
const createPublication

…

View on GitHub

Rate Limiting

GitHub Link: https://github.com/leaonline/ratelimit-factory
Packosphere: https://packosphere.com/leaonline/ratelimit-factory

Every time you use Methods and Publications you should use Meteor's DDP rate limiter to prevent overloading server resources by massive calls to heavy methods or publications.

You can also read more on rate limiting in the official Meteor docs.

We provide with our ratelimiter factory a fast and effective way to include ratelimiting to your Methods, Publications and Meteor internals:

$ meteor add leaonline:ratelimit-factory

Then add the Method or Publication definitions to the rateLimiter:

import { Todos } from '/path/to/Todos'
import { 
  runRateLimiter,
  rateLimitMethod,
  rateLimitPublication
} from 'meteor/leaonline:ratelimit-factory'

// ...

Object.values(Todos.publications).forEach(options => {
  createPublication(options)
  rateLimitPublication(options)
})

Object.values(Todos.methods).forEach(options => {
  createMethod(options)
  rateLimitMethod(options)
})

runRateLimiter(function (reply, input) {
  // if the rate limiter has forbidden a call
  if (!reply.allowed) {
    const data = { ...reply, ...input }
    console.error('rate limit exceeded', data)
  }
})

Under the hood the whole package uses DDPRateLimiter so you can also add numRequests and timeInterval to your Method or Publication definitions to get more fine-grained rate limits.

HTTP endpoints

GitHub Link: https://github.com/leaonline/http-factory
Packosphere: https://packosphere.com/leaonline/http-factory

Creating HTTP endpoints is possible in Meteor but it operates at a very low-level, compared to Methods or Publications and can become very cumberstone with a growing code complexity.

At lea.online we tried to abstract this process to make it similar to defining Methods or Publications in a rather descriptive way (as shown in the above sections). The result is our HTTP-Factory:

$ meteor add leaonline:http-factory
$ meteor npm install --save body-parser

import { WebApp } from 'meteor/webapp'
import { createHTTPFactory } from 'meteor/leaonline:http-factory'
import bodyParser from 'body-parser'

WebApp.connectHandlers.urlEncoded(bodyParser /*, options */) // inject body parser

export const createHttpRoute = createHTTPFactory({ 
  schemaFactory: definitions => new SimpleSchema(definitions)
 })

Now let's define an HTTP endpoint on our Todos:

Todos.routes = {}

Todos.routes.allPublic = {
  path: '/todos/public',
  method: 'get',
  schema: {
    limit: {
      type: Number,
      optional: true,
      min: 1
    }
  },
  run: onServer(function (req, res, next) {
    // use the api to get data instead if req
    const { limit = 15 } = this.data()
    return getCollection(Todos)
      .find({ isPublic: true }, { limit })
      .fetch()
  })
}

Creating the endpoints at startup is, again, as easy as with the other factories:

import { Todos } from '/path/to/Todos'
import { createHttpRoute } from '/path/to/createHttpRoute'

Object.values(Todos.routes).forEach(options => createHttpRoute(options))

Calling the endpoint can be done with fetch, classic XMLHttpRequest or Meteor's http library (used in the example):

import { Todos } from '/path/to/Todos'

HTTP.get(Todos.routes.allPublic.path, { params: { limit: 5 }}, (err, res) => {
  console.log(res.content) // [{...}, {...}, {...},{...}, {...}]
})

This is just a very small snipped of what you can do with this package! Read more about the API and documentation in the GitHub repository:

leaonline / http-factory

Create Meteor connect HTTP middleware. Lightweight. Simple.

Meteor HTTP Factory

Create Meteor WebApp (connect) HTTP middleware. Lightweight. Simple.

With this package you can define factory functions to create a variety of Meteor HTTP routes Decouples definition from instantiation (also for the schema) and allows different configurations for different types of HTTP routes.

Minified size < 2KB!

Why do I want this?

Decouple definition from instantiation
Easy management between own and externally defined middleware on a local or global level
Validate http request…

View on GitHub

Files and GridFs

GitHub Link: https://github.com/leaonline/grid-factory
Packosphere: https://packosphere.com/leaonline/grid-factory

Meteor has no builtin concept to upload Files but there are great packages out there, suche as Meteor-Files (ostrio:files).

It supports uploading files to several storages, such as FileSystem, S3, Google Drive or Mongo's Builtin GridFs. The last one is a very tricky solution but provides a great way to upload files to the database without further need to register (and pay for) an external service or fiddling with paths and filesystem constraints.

Fortunately we created a package with complete GridFs integration for Meteor-Files:

$ meteor add leaonline:files-collection-factory ostrio:files
$ meteor npm install --save mmmagic mime-types # optional

The second line is optional, these

import { MongoInternals } from 'meteor/mongo'
import { createGridFilesFactory } from 'meteor/leaonline:grid-factory'
import { i18n } from '/path/to/i8n'
import fs from 'fs'

const debug = Meteor.isDevelopment
const i18nFactory = (...args) => i18n.get(...args)
const createObjectId = ({ gridFsFileId }) => new MongoInternals.NpmModule.ObjectID(gridFsFileId)
const bucketFactory = bucketName => 
  new MongoInternals.NpmModule.GridFSBucket(MongoInternals.defaultRemoteCollectionDriver().mongo.db, { bucketName })
const defaultBucket = 'fs' // resolves to fs.files / fs.chunks as default
const onError = error => console.error(error)

export const createFilesCollection = createGridFilesFactory({ 
  i18nFactory, 
  fs, 
  bucketFactory, 
  defaultBucket, 
  createObjectId, 
  onError, 
  debug 
})

Now let's assume, our Todos app will have multiple users collaborating on lists and we want them to provide a profile picture then we can create a new FilesCollection with GridFs storage like so:

const ProfileImages = createFilesCollection({
  collectionName: 'profileImages',
  bucketName: 'images', // put image collections in the 'images' bucket
  maxSize: 3072000, // 3 MB max in this example
  validateUser: function (userId, file, type, translate) {
    // is this a valid and registered user?
    if (!userId || Meteor.users.find(userId).count() !== 1) {
      return false
    }

    const isOwner = userId === file.userId
    const isAdmin = ...your code to determine admin
    const isAllowedToDownload =  ...other custom rules

    if (type === 'upload') {
      return Roles.userIsInRole(userId, 'can-upload', 'mydomain.com') // example of using roles
    }

    if (type === 'download') {
      return isOwner || isAdmin || isAllowedToDownload // custom flags
    }

    if (type === 'remove') {
     // allow only owner to remove the file
     return isOwner || isAdmin
    }

    throw new Error(translate('unexpectedCodeReach'))
  }
})

With this short setup you will save lots of time and effort that you would waste, when trying to get this whole GridFs setup running.

leaonline / grid-factory

Simple factory to create FilesCollections

Meteor Grid-Factory

Create FilesCollections with integrated GridFS storage Lightweight. Simple.

With this package you can easily create multiple ostrio:files collections (FilesCollections) that work with MongoDB's GridFS system out-of-the-box.

Background / reasons

It can be a real hassle to introduce gridFS as storage to your project. This package aims to abstract common logic into an easy and accessible API while ensuring to let you override anything in case you need a fine-tuned custom behavior.

The abtract factory allows you to create configurations on a higher level that apply to all your FilesCollections, while you still can fine-tune on the collection level. Supports all constructor arguments of FilesCollection.

View on GitHub

A composition of all tools

The great thing about all these tools is, that they can easily be combined into a single pipeline while the several definitions control, what is actually to be created:

import { createCollection } from 'path/to/createCollection'
import { createFilesCollection } from 'path/to/createFilesCollection'
import { createMethod } from 'path/to/createMethod'
import { createPublication } from 'path/to/createPublication'
import { createHttpRoute } from 'path/to/createHttpRoute'

export const createBackend = definitions => {
  const collection = createCollection(definitions)

  // files collections could be indicated by a files property
  if (definitions.files) {
    createFilesCollection({ collection, ...definition.files })
  }

  // there will be no op if no methods are defined
  Object.values(definitions.methods || {}).forEach(options => {
    createMethod(options)
    rateLimitMethod(options)
  })

  // there will be no op if no publications are defined
  Object.values(definitions.publications || {}).forEach(options => {
    createMethod(options)
    rateLimitMethod(options)
  })

  // there will be no op if no publications are defined
  Object.values(definitions.routes || {}).forEach(options => {
    createRoute(options)
  })
}

Once you have setup this pipeline you can pass in various definitions, similar to the Todos object. The benefits of this approach will become more visible, once your application grows in terms of collections, methods and publications.

Some final notes

At lea.online we always try to improve our published code where we can. If you find any issues with the code in this article then please leave a comment and if you have trouble with the packages or miss crucial features then leave an issue in the repositories.

We hope these tools will help you boosting your productivity! 🚀

I regularly publish articles here on dev.to about Meteor and JavaScript. If you like what you are reading and want to support me, you can send me a tip via PayPal.

You can also find (and contact) me on GitHub, Twitter and LinkedIn.

Keep up with the latest development on Meteor by visiting their blog and if you are the same into Meteor like I am and want to show it to the world, you should check out the Meteor merch store.

Code splitting

Bonus: create an isomorphic wrapper

When to use which one?

Collection factory

Use-case

Collection instances

Meteor-Community-Packages / mongo-collection-instances

🗂 Meteor package allowing Mongo Collection instance lookup by collection name

Meteor Mongo Collection Instances

Installation

Usage Example

API

Mongo.Collection.get('name', [options])

Mongo.Collection.getAll()

Multiple connections

Method Factory

Adding constraints via mixins

leaonline / method-factory

Create validated Meteor methods. Lightweight. Simple.

Meteor ValidatedMethod Factory

Why do I want this?

Installation

Usage

Publication Factory

leaonline / publication-factory

Create Meteor publications. Lightweight. Simple.

Meteor Publication Factory

Why do I want this?

Installation

Usage

Rate Limiting

HTTP endpoints

leaonline / http-factory

Create Meteor connect HTTP middleware. Lightweight. Simple.

Meteor HTTP Factory

Table of Contents

Why do I want this?

Files and GridFs

leaonline / grid-factory

Simple factory to create FilesCollections

Meteor Grid-Factory

Background / reasons

Table of Contents

A composition of all tools

Some final notes

Read next

AI-Powered Code Generation: Revolutionizing Development

Build Live Code Editor Clone with HTML, CSS and JavaScript

Horóscopo com IA: Uma Experiência com Next.js e Gemini

JavaScript vs. Deno, Vite 6, Astro 5, and more