7 min read

Preventing offline data loss with Web Workers

The story

Your carefully crafted web app has gone offline and now the code kicks in to manage the data in the browser while the user is offline waiting for them to get back in range.

Time goes on and the user had already noticed a performance hit initially when they lost internet but now data is accumulating and the app quickly becomes a mess of UI lock up periods and choppy scrolling.

It's time to explore multi-threading your JavaScript with Web Workers.

Why I'm writing this

Whether I am working on a fun canvas game, a simple content app (like a to do list for example) I like to introduce a Web Worker at the earliest stages to take control of anything that the main UI thread doesn't need to do. This type of thinking is natural to a native App developer but an every JavaScript developer or one focussed on a backend language like Ruby, Java, .NET, or PHP doesn't want to acknowledge the browser has multiple threads available. It's been 10 years already, get your act together peeps.. It's embarrassing..
I've interviewed and worked with many developers do not seem to acknowledge or care about this type of code separation, and they've all never used the App they've developed as a user themselves, right? I can assure you if they had (and they likely have) they would feel like they could never brag about the website to their friends or next employer.

Hey, web and front-end JavaScript developers - be a user for a day, see the UI lock up and USE Web Workers already

Take some pride in your own work, don't just be proud of your clever syntax arrangements - that's not art! It's plain ignorant vanity.

Haven't heard of Web Workers yet?

I hope you've just started school or decided to dig into some JavaScript this year after learning some HTML and CSS because all browser vendors had fully implemented Web Workers the way they work today back in 2008. And they were available up to 2 years before that in some browsers.

It is 2016 and some of the more advanced among you have heard about the fantastic advancements in offline capabilities using Service Workers, and may have skimmed on the Web Workers (shame on you) or just found it plain confusing. Well It's quite simple what the differences are.

A Service Worker is not a Web Worker.

A service worker is a thread apart from the main window, you know the part with the DOM, every browser tab and window operates it's own thread with each DOM. A service worker has been exposed to you so that you may access all the network calls coming out of your main thread with your DOM. Thing about an <img tag with an src attribute. The service worker implements the ability for you as a developer to catch the network attempt that would fetch the image from the location in the src and do something, like cache the image so you only need to download it once, or, reply back to the browser with a base64 encoded image if the device has no internet connection awaiting the moment internet is available. Or you could real clever and prevent unauthorised access to resources before they even start to contact the server in the first place.

A Web Worker Is much simpler, though also a thread apart from the main browser window or tab, and able to make AJAX like network requests, it doesn't offer a developer any deeper access to the usual networking or caching features of a browser. It's uses are more varied, to the point, you use web workers to do all the heavy lifting on the client side so that the processing of such things do not affect the UI or UX in terms of animations, scrolling, drag'n'drop, ect. By setting up your process intensive logic via a web worker you ensure that the UI will feel as snappy and act as fluid as though you were not running any JavaScript at all.
A very important consideration on netbooks, tablet, low end mobile phones, and a massive host of users around the world visiting your site on slow internet (cough Australia..)

Web Worker by example

Let's create a way to autosave some page HTML as content along with it's arbitrary data such as name and category for example.

This is a fairly straightforward implementation;

index.html

<html>
  <head>
    <!-- basics -->
  </head>
  <body>
    <select class="autosave">
      <option disabled>- Saved Items -</option>
    </select>
  </body>
</html>

autosave.js

if (!!window.Worker) {
  Autosave = function () {
    var that = this;
    this.Worker = new Worker('/autosave.worker.js');
    this.Worker.addEventListener('message', function (e) {
      if (e.data === 'Database initialised.') {
        console.log(e.data);
      }
      if (e.data.action === 'add') {
        that.addItem((new Date(e.data.result.created)).toLocaleTimeString() + ' [' + e.data.result.category + '] ' + e.data.result.name, e.data.result.id);
      }
      if (e.data.action === 'list') {
        e.data.result.forEach(function(v){
          that.addItem((new Date(v.created)).toLocaleTimeString() + ' [' + v.category + '] ' + v.name, v.id);
        });
      }
    });
    this.Worker.postMessage({});
  };
  Autosave.prototype.save = function (category, name, content) {
    this.Worker.postMessage({
      action: 'add', params: {
        category: category,
        name: name,
        created: +new Date(),
        content: content
      }
    });
  };
  Autosave.prototype.list = function () {
    this.Worker.postMessage({action: 'list'});
  };
  Autosave.prototype.get = function (id) {
    this.Worker.postMessage({action: 'get', params: {id: parseInt(id)}});
  };
  Autosave.prototype.addItem = function (label, value) {
    //example adding a new option to the top of a select in the DOM
    $('select.autosave').children().eq(0).after(new Option(label, value));
  };
}

autosave.worker.js

(function () {
  "use strict";
  var dbName    = 'autosave',
      dbVersion = 1;

  importScripts('store.js');
  self.Store = new Store(self, dbName, dbVersion);

  self.onmessage = function (e) {
    var data     = e.data,
        action   = data.action,
        params   = data.params || {};

    if (action && self.Store[action]) {
      self.Store[action].call(self.Store, params, function (result) {
        self.postMessage({action: action, params: params, result: result});
      });
    }
  };
})();

store.js

Store = function (worker, name, version) {
  var that = this;
  this.name = name;
  this.version = version || 1;
  // Let us open our database
  var DBOpenRequest = worker.indexedDB.open(name, version);
  // these two event handlers act on the database being opened successfully, or not
  DBOpenRequest.onerror = function (e) {
    worker.postMessage('Error loading database.');
  };
  DBOpenRequest.onsuccess = function (e) {
    worker.postMessage('Database initialised.');
    // store the result of opening the database in the instance variable. This is used a lot below
    that.instance = DBOpenRequest.result;
  };
  // This event handles the event whereby a new version of the database needs to be created
  // Either one has not been created before, or a new version number has been submitted via the
  // indexedDB.open line above
  //it is only implemented in recent browsers
  DBOpenRequest.onupgradeneeded = function (e) {
    var instance = e.target.result;
    instance.onerror = function (e) {
      worker.postMessage('Error loading database.');
    };
    // Create an objectStore for this database
    var objectStore = instance.createObjectStore(name, {keyPath: 'id', autoIncrement: true});
    // define what data items the objectStore will contain
    objectStore.createIndex("name", "name", {unique: false});
    objectStore.createIndex("category", "category", {unique: false});
    objectStore.createIndex("created", "created", {unique: false});
    objectStore.createIndex("content", "content", {unique: false});

    worker.postMessage('Object store created.');
  };
  return this;
};
Store.prototype = {
  add: function (params, callback) {
    var that = this,
        newItem = {};
    // open a read/write db transaction, ready for adding the data
    var transaction = that.instance.transaction([that.name], "readwrite");
    // report on the success of opening the transaction
    transaction.oncomplete = function () {
      console.log('Transaction completed: database add finished.');
      typeof callback === 'function' && callback.call(params, params);
    };
    transaction.onerror = function () {
      console.log('Transaction not opened due to error: ' + transaction.error + '');
    };
    // call an object store that's already been added to the database
    var objectStore = transaction.objectStore(that.name);
    // add our newItem object to the object store
    var objectStoreRequest = objectStore.add(params);
    objectStoreRequest.onsuccess = function (e) {
      // report the success of our new item going into the database
      params.id = e.target.result;
    };
  },
  get: function (params, callback) {
    var that = this,
        id = params.id,
        result;
    // open a read/write db transaction, ready for adding the data
    var transaction = that.instance.transaction([that.name], "readwrite");
    // report on the success of opening the transaction
    transaction.oncomplete = function () {
      console.log('Transaction completed: database get finished.');
      typeof callback === 'function' && callback.call(params, result);
    };
    transaction.onerror = function () {
      console.log('Transaction not opened due to error: ' + transaction.error + '');
    };
    // call an object store that's already been added to the database
    var objectStore = transaction.objectStore(that.name);
    // Open our object store and then get a cursor list of all the different data items in the IDB to iterate through
    objectStore.openCursor().onsuccess = function (e) {
      var cursor = e.target.result;
      // if there is still another cursor to go, keep runing this code
      if (cursor) {
        // if there are no more cursor items to iterate through, say so, and exit the function
        if (cursor.value.id === id) {
          result = cursor.value;
          return;
        } else {
          // continue on to the next item in the cursor
          cursor.continue();
        }
      }
    };
  },
  truncate: function (params, callback) {
    var that = this;
    // open a read/write db transaction, ready for adding the data
    var transaction = that.instance.transaction([that.name], "readwrite");
    // report on the success of opening the transaction
    transaction.oncomplete = function () {
      console.log('Transaction completed: database truncate finished.');
      typeof callback === 'function' && callback.call(params);
    };
    transaction.onerror = function () {
      console.log('Transaction not opened due to error: ' + transaction.error + '');
    };
    // call an object store that's already been added to the database
    transaction.objectStore(that.name).clear();
  },
  update: function (params, callback) {
    var that = this;
    // open a read/write db transaction, ready for adding the data
    var transaction = that.instance.transaction([that.name], "readwrite");
    // report on the success of opening the transaction
    transaction.oncomplete = function () {
      console.log('Transaction completed: database update finished.');
      typeof callback === 'function' && callback.call(params, params);
    };
    transaction.onerror = function () {
      console.log('Transaction not opened due to error: ' + transaction.error + '');
    };
    // call an object store that's already been added to the database
    transaction.objectStore(that.name).put(params);
  },
  delete: function (params, callback) {
    var that = this,
        id = params.id;
    // open a read/write db transaction, ready for adding the data
    var transaction = that.instance.transaction([that.name], "readwrite");
    // report on the success of opening the transaction
    transaction.oncomplete = function() {
      console.log('Transaction completed: database delete finished.');
      typeof callback === 'function' && callback.call(params);
    };
    transaction.onerror = function () {
      console.log('Transaction not opened due to error: ' + transaction.error + '');
    };
    // call an object store that's already been added to the database
    transaction.objectStore(that.name).delete(id);
  },
  list: function (params, callback) {
    var that  = this,
        items = [];
    // open a read/write db transaction, ready for adding the data
    var transaction = that.instance.transaction([that.name], "readwrite");
    // report on the success of opening the transaction
    transaction.oncomplete = function () {
      console.log('Transaction completed: database list finished.');
    };
    transaction.onerror = function () {
      console.log('Transaction not opened due to error: ' + transaction.error + '');
    };
    // call an object store that's already been added to the database
    var objectStore = transaction.objectStore(that.name);
    // Open our object store and then get a cursor list of all the different data items in the IDB to iterate through
    objectStore.openCursor().onsuccess = function (e) {
      var cursor = e.target.result;
      // if there is still another cursor to go, keep runing this code
      if (cursor) {
        items.push(cursor.value);
        // continue on to the next item in the cursor
        cursor.continue();
        // if there are no more cursor items to iterate through, say so, and exit the function
      } else {
        typeof callback === 'function' && callback.call(params, items);
      }
    };
  }
};

And there you have it, an autosave implementation using web workers.

I hope this article helps you and saves you time - please spread the knowledge!