Function: Shipping Notifier

    +

    Goal: Send notifications when an order scheduled to arrive, when it is shipped, and when it is delivered.

    • This function shippingNotifier demonstrates a shipping workflow.

    • Requires Eventing Storage (or metadata collection), and "active", "archive", "notify" collections.

    • Requires four buckets metadata, active (the source), archive, and a notify.

      • The "notify"collection can be used to integrate with SDK or Kafka to send notifications.

    • Will operate on any doc with type === "ship".

    • Will update the source document with key information on each notify.

    • Delivered the shipping recored is archived.

    • On each notify data is read from the "active" bucket for type === "cust" and type === "order" as needed to build the notification.

    Other:

    • Note we could have used curl() to send the notify messages instead of using our staging bucket called "notify".

    • There are no try catch blocks and only limited error checking to highlight the functionality.

    • It is expected that the application that processes the "notify" bucket will purge the notification documents.

    • The notification history is stored in the shipping document and archived for all time.

    • shippingNotifier

    • Input Data/Mutation

    • Output Data/Mutation

    // To run configure the settings for this Function, shippingNotifier, as follows:
    //
    // Version 7.1+
    //   "Function Scope"
    //     *.* (or try bulk.data if non-privileged)
    // Version 7.0+
    //   "Listen to Location"
    //     bulk.data.active
    //   "Eventing Storage"
    //     rr100.eventing.metadata
    //   Binding(s)
    //    1. "binding type", "alias name...", "bucket.scope.collection", "Access"
    //       "bucket alias", "act_col",       "bulk.data.active",        "read and write"
    //       "bucket alias", "arc_col",       "bulk.data.archive",       "read and write"
    //       "bucket alias", "snd_col",       "bulk.data.notify",        "read and write"
    //
    // Version 6.X
    //   "Source Bucket"
    //     source
    //   "MetaData Bucket"
    //     metadata
    //   Binding(s)
    //    1. "binding type", "alias name...", "bucket",     "Access"
    //       "bucket alias", "act_col",       "active",     "read and write"
    //       "bucket alias", "arc_col",       "archive",    "read and write"
    //       "bucket alias", "snd_col",       "notify",     "read and write"
    
    function sendNotifySchedDelivCallback(context) {
        // This is a normal sceduled delivery notificaton
    
        // Look up the controlling ship: doc
        var shipkey = context.id;
        var shipdoc = act_col[shipkey];
        if (shipdoc === null) {
            // stale timer
            return;
        }
    
        // Make sure we are active and still need to send
        if (shipdoc.type != "ship" || !shipdoc.active || shipdoc.notifys[context.idx].notifySent) return;
    
        // Look up the realted order: doc
        var orderkey = "order" + ":" + shipdoc.id;
        var orderdoc = act_col[orderkey];
    
        // Look up the realted cusomer: doc
        var custkey = "cust" + ":" + orderdoc.cust_id;
        var custdoc = act_col[custkey];
        var notifyId = "ntfy" + ":" + context.idx + ":" + shipkey;
    
        // log('shipdoc',  shipdoc);
        // log('orderdoc', orderdoc);
        // log('custdoc',  custdoc);
        // log("notifyId",notifyId);
    
        var senddoc = {
            "notifyReason": context.item.notifyReason,
            "first_name": custdoc.first_name,
            "last_name": custdoc.last_name,
            "email": custdoc.email,
            "phone": custdoc.phone,
            "items": orderdoc.items,
            "utcOffset": shipdoc.utcOffset
        };
    
        // Add any special details
        if (context.item.notifyReason === "scheduled delivery") {
            senddoc["schedDelivTs"] = shipdoc.schedDelivTs;
        } else
        if (context.item.notifyReason === "delivered") {
            senddoc["deliveredTs"] = shipdoc.deliveredTs;
        } else
        if (context.item.notifyReason === "shipped") {
            senddoc["shippedTs"] = shipdoc.shippedTs;
        }
    
        // Write to send bucket -or- emit via cURL
        snd_col[notifyId] = senddoc;
    
        // Mark as sent
        shipdoc.notifys[context.idx].notifySent = true;
    
        // See if we are done and can archive this
        if (shipdoc.delivered && context.item.notifyReason === "delivered") {
            shipdoc.active = false;
            // Yes we can archive write to archive bucket
            arc_col[shipkey] = shipdoc;
            // and remove
            delete act_col[shipkey];
        } else {
            // No just update in the source bucket
            act_col[shipkey] = shipdoc;
        }
    
        log("senddoc", senddoc);
    }
    
    function OnUpdate(doc, meta) {
        // Filter out non interesting items
        if (doc.type != "ship" || !meta.id.startsWith("ship:") || !doc.active) return;
    
        var nowMs = Date.now();                // this instant or now in ms.
        var nowSec = Math.trunc(nowMs / 1000); // this instant or now in sec.
    
        if (doc.shipped || doc.delivered) {
            // these are events they do not need to be scheduled via a Timer
            if (doc.shipped) {
                if (doc.shippedTs === null) {
                    doc.shippedTs = nowSec;
                }
                var item = {
                    "notifyReason": 'shipped',
                    "notifyTs": nowSec,
                    "notifySent": false
                };
            }
            if (doc.delivered) {
                if (doc.deliveredTs === null) {
                    doc.deliveredTs = nowSec;
                }
                var item = {
                    "notifyReason": 'delivered',
                    "notifyTs": nowSec,
                    "notifySent": false
                };
            }
            // Add to the notification array or history
            doc.notifys.push(item);
            // Write the source doc since we will sending an immediate notification
            act_col[meta.id] = doc;
            var context = {
                "item": item,
                "idx": doc.notifys.length - 1,
                "id": meta.id
            };
            // There no need for a timer we can do this now since it is an event
            sendNotifySchedDelivCallback(context);
            return;
        }
    
        // Look for any needed notifications in the future
        for (var idx = 0; idx < doc.notifys.length; idx++) {
            var item = doc.notifys[idx];
            if (!item.notifySent) {
    
                // JavaScript works in ms. BUT the doc's fields are in sec. - so convert and make a Date()
                var fireAt = new Date(item.notifyTs * 1000);
    
                // Make unique ref for this notification can overwrite/adjust or cancel
                var notifyId = "ntfy" + ":" + idx + ":" + meta.id;
    
                // Pass minimal data in our context, the callback will look everything else up.
                var context = {
                    "item": item,
                    "idx": idx,
                    "id": meta.id
                };
    
                // We will always 'overwrite' this timer(s) notification by the Timer's
                // reference_id (6.6.0+ required for this) on every mutation
                // log("create/overwrite notification "+ notifyId, item);
                createTimer(sendNotifySchedDelivCallback, fireAt, notifyId, context);
            }
        }
    }

    We want to create a test set of three (3) documents. Use the Query Editor to insert the the data items (you do not need an index).

    For key "ship:dea0fca2-e7b7-11ea-adc1-0242ac120002", you may want to adjust the timestamps as the times are in seconds since Unix epoch. Use a tool like https://www.dcode.fr/timestamp-converter or https://www.epochconverter.com/ .
      UPSERT INTO `bulk`.`data`.`active` (KEY,VALUE)
      VALUES ( "order:dea0fca2-e7b7-11ea-adc1-0242ac120002", {
        "type": "order",
        "id": "dea0fca2-e7b7-11ea-adc1-0242ac120002",
        "cust_id": 108998,
        "items": [
          {
              "sku": "SK18768",
              "descr": "Ticondorna pencils 12 pack",
              "qty": 3
          },
          {
              "sku": "SK89736",
              "descr": "Sharpie large marker",
              "qty": 1
          }
        ]
      }),
      VALUES ( "cust:108998", {
        "type": "cust",
        "id": 108998,
        "first_name": "John",
        "last_name":  "Smith",
        "email": "jon.smith@gmail.com",
        "addr1": "1010 E. 100th Ave.",
        "addr2": "Apt 101B",
        "city": "New York",
        "state": "NY",
        "zip": 10000,
        "phone": "+1 714-222-2222"
      }),
      VALUES ( "ship:dea0fca2-e7b7-11ea-adc1-0242ac120002", {
        "type": "ship",
        "id": "dea0fca2-e7b7-11ea-adc1-0242ac120002",
        "utcOffset": -420,
        "orderTs": 1598214610,
        "schedDelivTs": 1598486400,
        "shippedTs": null,
        "deliveredTs": null,
        "notifys": [
          {
            "notifyTs": 1598450400,
            "notifyReason": "scheduled delivery",
            "notifySent": false
          }
        ],
        "exceptions": [],
        "shipped": false,
        "delivered": false,
        "active": true
      });

    To fully exercise the logic, run the following steps(to re-run flush the 'active', 'archive' and 'notify' collections and redo the UPSERT the data):

    • Deploy the Function with a Feed Boundary from "Everything".

      • Wait for about 7-14 seconds (timers are high volume not wall clock accurate) and notice collection "notify" has our first notification (the timer was scheduled in the past).

      • The shipping document will be modified in collection 'active' as follows:

        UPDATED/OUTPUT: KEY ship:dea0fca2-e7b7-11ea-adc1-0242ac120002 in collection "active"
        {
          "active": true,
          "delivered": false,
          "deliveredTs": null,
          "exceptions": [],
          "id": "dea0fca2-e7b7-11ea-adc1-0242ac120002",
          "notifys": [
            {
              "notifyReason": "scheduled delivery",
              "notifySent": true,
              "notifyTs": 1598450400
            }
          ],
          "orderTs": 1598214610,
          "schedDelivTs": 1598486400,
          "shipped": false,
          "shippedTs": null,
          "type": "ship",
          "utcOffset": -420
        }
      • You will now have the first notificaton document in collection 'notify' as follows:

        NEW/OUTPUT: KEY ntfy:0:ship:dea0fca2-e7b7-11ea-adc1-0242ac120002 in collection "notify"
        {
          "notifyReason": "scheduled delivery",
          "first_name": "John",
          "last_name": "Smith",
          "email": "jon.smith@gmail.com",
          "phone": "+1 714-222-2222",
          "items": [
            {
              "descr": "Ticondorna pencils 12 pack",
              "qty": 3,
              "sku": "SK18768"
            },
            {
              "descr": "Sharpie large marker",
              "qty": 1,
              "sku": "SK89736"
            }
          ],
          "utcOffset": -420,
          "schedDelivTs": 1598486400
        }
      • The application log for the Eventing handler will show something like the following:

        2021-07-18T21:17:51.715-07:00 [INFO] "senddoc" {"notifyReason":"scheduled delivery","first_name":"John","last_name":"Smith","email":"jon.smith@gmail.com","phone":"+1 714-222-2222","items":[{"descr":"Ticondorna pencils 12 pack","qty":3,"sku":"SK18768"},{"descr":"Sharpie large marker","qty":1,"sku":"SK89736"}],"utcOffset":-420,"schedDelivTs":1598486400}

    • In collection "active" mutate ship:dea0fca2-e7b7-11ea-adc1-0242ac120002 by setting "shipped" to true.

      • The shiping document will be automatically modified in collection 'active' as follows:

        UPDATED/OUTPUT: KEY ship:dea0fca2-e7b7-11ea-adc1-0242ac120002 in collection "active"
        {
          "active": true,
          "delivered": false,
          "deliveredTs": null,
          "exceptions": [],
          "id": "dea0fca2-e7b7-11ea-adc1-0242ac120002",
          "notifys": [
            {
              "notifyReason": "scheduled delivery",
              "notifySent": true,
              "notifyTs": 1598450400
            },
            {
              "notifyReason": "shipped",
              "notifyTs": 1626668498,
              "notifySent": true
            }
          ],
          "orderTs": 1598214610,
          "schedDelivTs": 1598486400,
          "shipped": true,
          "shippedTs": 1626668498,
          "type": "ship",
          "utcOffset": -420
        }
      • You will now have the second notificaton document in collection 'notify' as follows:

        NEW/OUTPUT: KEY ntfy:1:ship:dea0fca2-e7b7-11ea-adc1-0242ac120002 in collection "notify"
        {
          "notifyReason": "shipped",
          "first_name": "John",
          "last_name": "Smith",
          "email": "jon.smith@gmail.com",
          "phone": "+1 714-222-2222",
          "items": [
            {
              "descr": "Ticondorna pencils 12 pack",
              "qty": 3,
              "sku": "SK18768"
            },
            {
              "descr": "Sharpie large marker",
              "qty": 1,
              "sku": "SK89736"
            }
          ],
          "utcOffset": -420,
          "shippedTs": 1626668498
        }
      • The Application log for the Eventing handler will show something like the following

        2021-07-18T21:21:38.547-07:00 [INFO] "senddoc" {"notifyReason":"shipped","first_name":"John","last_name":"Smith","email":"jon.smith@gmail.com","phone":"+1 714-222-2222","items":[{"descr":"Ticondorna pencils 12 pack","qty":3,"sku":"SK18768"},{"descr":"Sharpie large marker","qty":1,"sku":"SK89736"}],"utcOffset":-420,"shippedTs":1626668498}

    • In collection "active", mutate ship:dea0fca2-e7b7-11ea-adc1-0242ac120002 again by setting "delivered" to true.

      • The shiping document will be removed from collections 'active' and archived to the collection 'archive' as follows:

        DELETE/OUTPUT: KEY ship:dea0fca2-e7b7-11ea-adc1-0242ac120002 in collection "active"
        
        NEW/OUTPUT: KEY ship:dea0fca2-e7b7-11ea-adc1-0242ac120002 in collection "archive"
        {
          "active": false,
          "delivered": true,
          "deliveredTs": 1626668622,
          "exceptions": [],
          "id": "dea0fca2-e7b7-11ea-adc1-0242ac120002",
          "notifys": [
            {
              "notifyReason": "scheduled delivery",
              "notifySent": true,
              "notifyTs": 1598450400
            },
            {
              "notifyReason": "shipped",
              "notifyTs": 1626668498,
              "notifySent": true
            },
            {
              "notifyReason": "delivered",
              "notifyTs": 1626668622,
              "notifySent": true
            }
          ],
          "orderTs": 1598214610,
          "schedDelivTs": 1598486400,
          "shipped": true,
          "shippedTs": 1626668498,
          "type": "ship",
          "utcOffset": -420
        }
      • You will now have the third and final notificaton document in collection 'notify' as follows:

        NEW/OUTPUT: KEY ntfy:2:ship:dea0fca2-e7b7-11ea-adc1-0242ac120002 in collection "notify"
        {
          "notifyReason": "delivered",
          "first_name": "John",
          "last_name": "Smith",
          "email": "jon.smith@gmail.com",
          "phone": "+1 714-222-2222",
          "items": [
            {
              "descr": "Ticondorna pencils 12 pack",
              "qty": 3,
              "sku": "SK18768"
            },
            {
              "descr": "Sharpie large marker",
              "qty": 1,
              "sku": "SK89736"
            }
          ],
          "utcOffset": -420,
          "deliveredTs": 1626668622
        }
      • The Application log for the Eventing handler will show something like the following

        2021-07-18T21:23:42.248-07:00 [INFO] "senddoc" {"notifyReason":"delivered","first_name":"John","last_name":"Smith","email":"jon.smith@gmail.com","phone":"+1 714-222-2222","items":[{"descr":"Ticondorna pencils 12 pack","qty":3,"sku":"SK18768"},{"descr":"Sharpie large marker","qty":1,"sku":"SK89736"}],"utcOffset":-420,"deliveredTs":1626668622}

    Note that with respect to the notifications that were created:

    • index 0 created a Timer that was fired immediately as it used a timer and was in the past.

      notifyTs = 2020-08-26T14:00:00.000Z or Wed Aug 26 2020 07:00:00 GMT-0700 (Pacific Daylight Time)

    • index 1 was an event e.g. shipped was mutated to true (it didn’t need a Timer) and fired instantly.

      shippedTs = 2021-07-19T04:21:38.000Z or Sun Jul 18 2021 21:21:38 GMT-0700 (Pacific Daylight Time)

    • index 2 was an event e.g. delivered was mutated to true (it didn’t need a Timer) and fired instantly.

      deliveredTs = 2021-07-19T04:23:42.000Z or Sun Jul 18 2021 21:23:42 GMT-0700 (Pacific Daylight Time)