Programming the Burpple Beyond Tracker

Sep 25 2018

About the Project

I needed to destress from work by building something without any meetings, free from considerations of feasibility, usefulness or ROI.

Then Burpple Beyond launched.

So I built a service to track the discounts on Burpple Beyond that would notify me (and others in my Burpple Beyond Telegram Group) when promotions were added or removed from the subscription.

Looking for a Burpple Beyond promo code? Feel free to ask for one in my Burpple Beyond fans Telegram Group

Burpple Beyond Promo Code Banner

My Pain Points

If you haven’t heard of Burpple Beyond before, it’s a subscription plan run by Singapore start up Burpple that entitles you to redeem four 1-for-1 deals at each Beyond food partner in Singapore. Vouchers range from 1-for-1 buffets, lobsters, salads, and steak to 1-for-1 wanton mee and bak kut teh.

Think of it as The Entertainer, only the food vouchers, and at approximately half the price.

You browse and redeem these discounts primarily through the Burpple app.

Screenshot of the Burpple Beyond app

But when it launched, the app had a few pain points:

  • No search capability: There was no search box. The list of food partners was arranged alphabetically by default, with no other alternatives. This was particularly painful when deciding whether an option you were looking for had “The” prefixed to its name, and would therefore be hidden all the way down in “T” instead of something like “A”. (EDIT: Two months after launch, the list can now be sorted in alternative ways, but still not searched.)

  • Alphabetical sort by default: I did not think the alphabetical sort by default was ideal either. I would prefer a sort by nearness, by default, instead. (EDIT: This was added in a later update, to my delight. Great job, Burpple Beyond team!)

  • Slow to load: Loading the app was a pretty slow experience. This can’t really be helped, because it needs to 1) Start up the app 2) Load the default Burpple social network screen, before finally, 3) Loading the list of Burpple Beyond merchants, sorted, separately. But its a pain to use if you need to scroll through the list to decide on a place to eat.

  • No indication when offers are added or removed: I would see the total number of merchants go up and down with no inkling of what the details of the changes were. Burpple has since introduced a brightly coloured yellow tag to promote “Newly Added” food partners, as well as publishing monthly blog posts promoting “What’s new on Burpple Beyond”. But for business reasons I can respect, they aren’t yet announcing when offers are removed. Since I make plans to go to Beyond venues in advance, I would like to know if my plans have to change!

My Solution

So I built the Burpple Beyond Tracker, code-named licklilicky.

Screenshot of Burpple Beyond Tracker

Features

  • Search the full list of Burpple Beyond food partners, pretty instantly

  • Nearby sort by default

  • Fast performance! Text-only, easily scannable.

  • Includes filters for newly added or removed food partners

The service also includes a Burpple Beyond Telegram Group. At the end of each working day, I update the list of Burpple Beyond offers, and send a chat message of what’s changed to the group:

Sample response of my Telegram Chat Bot

Join 60 other Burpple Beyond fans in our Telegram Group.

Technical Implementation:

Tracking which Burpple Beyond deals are added or removed

The first step was to scrape relevant data from the Burpple Beyond app.

To do that, I needed to reverse engineer how the Burpple engineers are sending data to their app. I used a Charles Proxy to monitor network traffic from my iPhone. That’s when I discovered that the entire list of Burpple Beyond restaurants is essentially sent as a JSON with authentication in the URL as a param, and can be accessed from anywhere. The URL structure is something like this: https://example.com/venues.json?auth=MY_PERSONAL_AUTH_TOKEN

Here’s an example of the JSON received:

{
  data: [
    {
      id: 1,
      name: "Delicious Restaurant",
      formatted_price: "~$25/pax",
      location: {
        neighbourhood: "Tanjong Pagar",
        longitude: -481516,
        latitude: 2342
      },
      avatar: {
        small: "https://cdn.example.com/delicious.png?s=44",
        medium: "https://cdn.example.com/delicious.png?s=88",
        large: "https://cdn.example.com/delicious.png?s=132",
      }
    },
    {
      id: 2,
      ...
    }
  ]
}

On the assumption that IDs are unique and immutable, this made creating a diff of the current list and the previous JSON very straightforward. To get all newly added venues, I simply checked for any objects in the current list that weren’t in the previous list. To get all removed venues, I checked for any objects in the previous list not in the current list. I saved the timestamp when my script first noticed the object was added or removed, and I saved my own JSON list. You can find the script to diff the JSONs here (yes, it’s hardcoded to the Beyond JSON data structure).

Displaying the full list of Burpple Beyond deals

I didn’t really want to invest much effort building the frontend, beyond insisting that it had to be a very clean design and load quickly (text and mobile-first!). So I forked the design from Repokemon, and deployed the static site using Surge and Now (it helps to have mirrors!).

The design calls for flexbox to display all the venues side-by-side, which allows the venue items to be resized and rearranged as needed.

#list {
  display: flex;
  flex-wrap: wrap;
  justify-content: center;
}
#list li {
  display: inline-flex;
  flex-grow: 1;
  flex-shrink: 1;
  flex-basis: 120px;
  align-items: stretch;
}

Example of CSS backdrop-filter

Only if you’re on Safari, the bottom sticky bar that holds the filter controls uses a spiffy backdrop-filter: blur CSS effect, while relying on built-in CSS check for feature compatibility with the @supports syntax for everyone else:

#controls {
  -webkit-backdrop-filter: blur(5px);
  backdrop-filter: blur(5px);
}
@supports (not (backdrop-filter: blur(5px))) and (not (-webkit-backdrop-filter: blur(5px))) {
  #controls {
    background-color: rgba(255, 255, 255, 0.9);
  }
}

The Added Recently filter was easy to define: any venue with a added_timestamp within the last 24 hours (stored as JavaScript epoch time), since the tracker runs once every day just before dinner at 5:30pm SGT. The Removed filter is even easier: any venue with a removed_timestamp.

The Nearby filter is a lot more interesting, and I certainly learnt some things while building it.

Retrieving the User’s current location using browser Geolocation API

I’d never used the Geolocation API before, so this was exciting. Unfortunately, I still think the user experience is not great. I have to ask for permission every time I request for the device location, and it takes forever to get the location.

Example of Geolocation API in Firefox

Code-wise, it introduces a drop of async behaviour into what would otherwise be a pure cup of sync code. To make it easier to reason about, I opted to use the Promise syntax. Given how long the Geolocation API takes to return with the location, the sorted list is cached through a memoized variable (nearbyListCache) (something I first picked up through Ruby) for a speedier User Experience (for example, if the User clicks from Nearby to New, and back to Nearby again, there shouldn’t be another request for location).

Also, by saving it to a variable instead of updating the DOM directly in the success callback, I ensure that if the User clicks away to a different filter, the DOM won’t get updated the moment the nearby filter resolves, and the DOM remains at the User’s desired filter.

// pseudo-code
var nearbyListCache;
var filter;

document.querySelector('controls').on('change', function(e) {
  filter = e.target.name
  renderList()
})

function renderList () {
  if (filter === 'nearby') {
    if (nearbyListCache && nearbyListCache.length) {
      // Use the cache value if it exists
      htmlContent = nearbyListCache;
      return Promise.resolve()
    } else {
      return new Promise(function(resolve, reject) {
        navigator.geolocation.getCurrentPosition(resolve, reject, options)
      }).then(function(position) {
        // success callback function
        formattedData = data.sort(sortFuncs[filter])
        nearbyListCache = formattedData.map(convertListItemToHTML).join('');
        htmlContent = nearbyListCache;
        return Promise.resolve()
      }).catch(function(err) {
        htmlContent = err;
        return Promise.resolve()
      })
    }
  }.then(() => filter === 'nearby' ? updateDOMWith(htmlContent) : doNothing())
}

In the future, I hope to explore HTML5 Local Storage as an option for the nearbyListCache and the Beyond restaurant JSON.

Sorting the Nearby List

How do you figure out how far one point is away from another from just the logitude and latitude?

The Haversine formula.

It allows you to calculate the orthodromic distance, the shortest distance between two points on a sphere, as measured along the surface of the sphere (as opposed to a straight line through the sphere’s interior).

Great-circle Distance

If you’re interested in the exact details of the formula, you can check out the implementation I used in JavaScript.

To be honest, I have no idea how to implement a performant client-side JSON text search.

Thankfully, I was able to find a great library out there: Elasticlunr.

I just had to make sure that the search index is reset each time a filter is applied by the user.

Also, to prevent unnecessary searches before the User is done typing - to prevent any lag while the User is entering their search terms - I used a debounce of 300ms between keyup events before running the search function.

Finally, I had to check that when the search input is cleared, the original full filtered list is displayed once again, not just the search results.

Adding the Burpple Beyond Tracker to the Homescreen

I never use Lickilicky on the desktop, and I’d rather not have the web browser UI when I’m using it. Turns out, there’s ways to make a webapp look like a native app!

For iOS, these are the meta tags to use:

  • Icon: <link rel="apple-touch-icon" href="/custom_icon.png">

  • Title: <meta name="apple-mobile-web-app-title" content="AppTitle">

  • Hiding Safari UI: <meta name="apple-mobile-web-app-capable" content="yes">

  • Changing the Status Bar Appearance: <meta name="apple-mobile-web-app-status-bar-style" content="black">

Find out more in this Google Developer Guide to Add to Home Screen.

Unfortunately, because of the limitations of iOS, the UX is pretty terrible for a homescreen-ed webapp. iOS aggressively kills tasks in the background, such that switching away the webapp will almost certainly result in the task being killed. In lickilicky’s case, you are forced to re-approve use of location tracking, or the JSON of venues is redownloaded and your scroll and filter positions are lost. I intend to solve this by exploring LocalStorage or Webworkers in the future.

Updating the Tracker Regularly

Now that I had the frontend and tracking in place, my Burpple Beyond tracker webapp was essentially working.

In fact, for the first month, that’s all I had. Whenever I noticed that the merchant count in the Burpple Beyond app had changed, I would manually run my code to produce my diff of the Beyond promos and deploy the new version of the static site.

But the list of Beyond promotions started to change more rapidly, necessitating an automated deploy. I needed to find out how to schedule code to run regularly at certain times. Up to that moment, I’d only ever worked with the CI/CD service, TravisCI. Unfortunately, I’d found TravisCI’s scheduler lacking (only daily/weekly/etc options). So I picked up CircleCI, which would allow granular scheduling, just like a cron job. Right now, I am partial to CircleCI’s syntax and configuration flexibility, over the workaround of putting my CI specific stuff in separate bash shell script files when using Travis.

You can check out my CircleCI configuration if you’d like to see how to schedule a build. Side-note: If I’d only required the diff as an API, without the deploy of the frontend, I would have used a serverless set up, uploading the diff code as a Lambda function invoked through API Gateway, triggered by a Cloudwatch Alarm. The JSON list might have been stored on S3.

Sending Update Notifications to Telegram

The frontend is great - but thats really mainly for searching when I’m looking for a place to eat.

For the updates on which venues are added or removed to be truly useful, it couldn’t just live on the frontend - it had to be sent as a notification.

And I wanted to share this feature with others - hence the idea of a Telegram Supergroup.

In brief, here’s how I set it up:

  • Creating a Burpple Beyond Telegram Channel @burpplebeyond

  • Creating a Burpple Beyond Tracker Bot by talking to Telegram’s Botfather (instructions here)

  • With the Telegram Token and Channel ID, send a GET request to Telegram’s API whenever a difference in Beyond partners is found

  • Because of issues with special characters like * in Markdown parse mode and URL encoding, I used HTML parse mode instead for better formatted messages

I hope to enrich these messages even more in the future by including richer location information, cuisine category, and especially images. Let me know in the Github Issues of the Tracker what you’d like me to prioritise.

Sharing about the Burrple Beyond Tracker

I posted updates for a while on Hardwarezone, but given the lack of responses I’ve since stopped for now.

What’s Next for This Project

I really loved working on this. Not only was it silly and completely at my discretion, it turned out to be really useful to me and to others.

Working on this was also an exercise in agile methodology - I launched with a working frontend, but behind the scenes, a lot of manual reconciliation and deployment was taking place. Only once I’d validated that there were enough changes to the Beyond partners to warrant a Tracker did I invest the effort to automating everything and building the Telegram group.

The response - in the form of members who’ve joined the Burpple Beyond fans Telegram Group - has been heartening, and I hope to develop this Tracker into an ultimate 1-for-1 food deals comparison app for Singapore.


Jared Tong is a software engineer based in πŸ‡ΈπŸ‡¬ Singapore. You should follow him on Github

Questions?

Have a question about this post, web development, or anything else? Ask away on Telegram or in my AMA github.