At the rate Lob is growing our teams might not always make time to be creative or focus on something outside the roadmap. So twice a year, we force them to. Just kidding, our internal hackathon is always optional, but we certainly encourage all employees to participate, engineers and non-engineers alike. We believe hackathons are an opportunity to be inventive and do something outside the normal routine. As a company we want to embody one of our core values “Be Bold” and we encourage folks to push the envelope, pun intended (cuz you know, Lob plays in the direct mail space).
We also wanted to have a little fun with a theme and chose Back to the Future as our inspiration. This classic 80s movie series has stood the test of time and is a “totally awesome” representation of technology and innovation—like hoverboards and self-lacing sneakers—and the 80s! In addition to themed messaging (including Marty and Doc emojis in Slack) we had two project tracks:
Time Travel is Real: These projects deal with time, dates, or time-series data in some way. This includes time saving productivity hacks.
Roads? Where we’re going, we don’t need roads: Go off the beaten path (anything goes!) and turn Lob into a flying Delorean with your idea.
Our teams had just over 36 hours to tackle their projects, then they were asked to give a 5-minute presentation/demo. In typical hackathon-style, there was a last-minute scramble and some finger crossing. As our first presenter noted, “like a normal software engineer, I'm just not ready just yet. So, I'm going to do my best to just wing it from here.”
Eliminate repeating fixture data
In order to address the problem of repeating fixture data when writing tests for our api (the center of our universe), Karmbir Chima created tooling to provide a pre-bundled set of fixture data that can be stitched together and inserted into our testing database. Eliminating the dupes means better tests (“less lines of code mean fewer places for bugs to hide”) and the new process overall helps automate a very manual process, saving time. Doc Brown would approve of this productivity hack.
Improve account management
Myrtle Tzouma, Shanna Murry, Makeda Davis, Mocha Brown, and Hanqing Chen teamed up to tackle a customer service/billing challenge. Lob uses an internal dashboard called Lens for customer account management. If a user needs to remove a payment method on an account, there's not an easy way to do it in the UI. The payment method must be removed from the Stripe dashboard, and from our dashboard with a manual POST request (Postman). Adding a small trash can icon—and associated functionality—is never as easy as it seems. The team had to touch three repos to make it happen. But the result is the ability to easily remove payment methods in fewer and less complex steps, saving not only customer service, but also engineers (who frequently test in the dashboard/API) time and effort. While not as cool as Mr.Fusion, it’s a pretty useful little trash can.
Answers at your fingertips
The Address Verification team at Lob has found themselves answering some of the same questions over and over again in our internal Slack channel, asked by the team at large, and most often the on-call engineers. With arguably one of the best team names—FAQ to the Future—engineering manager Sherman Gore built a Slack bot to solve this problem. Built in Node.js and hosted by Glitch, the bot will answer the question and supply supporting documentation.
Capacity management
Some days we have welcome, but unpredictable, volume surges; as Lob continues to grow we must ensure these don’t stress our print partner network, or put our mail speed in jeopardy. What is currently a very tedious and manual process (think a flurry of phone calls) is moving towards automation thanks to Dave Krawczuk, Trent Blackwell, Anisha Biswaray, Michael Morgan, and Michael Tuffley. This team began the development of an environment for our print partners to be able to capture (and execute) additional volume. They demonstrated a portal that would show all mailpieces available (i.e. overflow not automatically routed by current logic in place), and a partner could claim them should they have additional (flux) capacity.
A new application for machine learning
During peak volume days, USPS congestion causes delayed mail pieces, putting our target delivery dates at risk. Team “Hack the Mailman” (Aditi Ramaswamy, Anisha Biswaray, and Luke Birdeau) came up with a creative solution. Their goal was to create a model that can identify patterns of congestion within Nation Distribution Centers (NDCs), and route mail to the fastest (or non-congested) NDC for more reliable and predictable service. This will allow us to better handle seasonality, and enhance logistic management with drop shipping.
We love the USPS, but as with any large entity, they are not immune to challenges; USPS tracking data can be a bit spotty. But with enough data and a little intelligence, we could fill in the gaps….and possibly predict the future. In summary, tracking data (500K unique data points) was retrieved from our historical data then after a lot of data wrangling, was fed into sklearn Random Forest regression. Then, it entered an ML flow invoked with a REST API call to allow for routing engine augmentation.
When all was said and done, the test data showed the closest partner isn't always the fastest partner, and the ML model can help Lob improve our routing. We are excited to expand the project and will only learn more from more data. Much like the Lob Confidence Score enhances our Address Verification API, so does this model for our routing engine. Given the potential positive impact on Lob’s operations (and that data from the past is being used to help predict the future), this team earned second place overall. They also deserve a shout out for their use of memes.
A new InDesign extension
An estimated 90% of our enterprise customers are using InDesign to create PDFs of their mailpieces (to send on to our print partners). Built as an MXI, Heath Horton on our Rendering/PDF Pipeline team developed a Lob extension for InDesign—a ZXP featuring easy-to-use templates—that once fleshed out, will be available on Adobe Exchange.
AsyncAPI
Gabriel Gore, Sachin Muralidhara, and Sean McGovern taught us a little more about AsyncAPI , an open-source specification for event-driven architectures (EDA). It defines interfaces for Async APIs, similar to what OpenAPI or Swagger does for RESTful services. You can automate and formalize docs, you can generate code, you can validate schemas for event-driven microservices, and it's compatible with OpenAPI for a lot of use cases. (If you have an existing code base with Open API schemas in it, you may be able to port those over to AsyncAPI; here’s a nifty comparison chart.)
The team used AsyncAPI to begin building a spec for events within Lob’s Campaigns product (a new product in beta to allow enterprise marketers to send direct mail at scale). This yields two benefits; both improve developer productivity. The first and most obvious is an easily-accessible API spec. The second is improved testing as a result of having a simple and scalable way to validate messages on either side of the broker. (The example given was running into issues where you’re writing a function that pulls off an SQSQ but have no idea what's actually on the message payload because there's no contract to reference.)
Given the newness of AsyncAPI there is not yet a lot of tooling available, and the SQS protocol bindings are still in development, so the team used a lightweight loT protocol called mqtt for proof-of-concept. Despite these challenges, the team was able to demonstrate success to an excited audience; this could have some very positive implications for development in a number of use cases at Lob.
Lob QR codes
Lob knows the value of hyper-personalized direct mail and our templates are designed to accommodate QR codes as part of this strategy. However, we’ve relied on the customers or outside providers to supply these…Until now. A small enterprising team of hackers decided to tackle this in-house and managed to build out end-to-end functionality for integrating QR codes into a campaign—and in doing so secured first prize. Congratulations to Muhammad Martinez, Samkit Sheth, Pat Thulen, Vishal Mangwani, and David Xu.
Showing off the backend, they demonstrated via Postman how one could input information like destination URL and number of mailpieces to create a campaign, for which Lob will then generate a unique QR code. Created as just another merge_variable (a dynamic field that can be updated within a mailpiece template) these codes will appear on each mailpiece rendered. When each QR is scanned it maps back to the campaign, providing analytics about each mailpiece (including the time and location of each scan) along with aggregate data about the campaign. So not only can Lob automate direct mail, it can now help measure its effectiveness. Great Scott!
Playing with livebook
Our resident Elixir SME, Guy Argo, worked with livebook (similar to Jupyter notebooks) to allow dynamic exploration of Lob’s Address Verification API. Or in other words, created animated maps that you could interact with; the example was following a car across town. A future application of this at Lob could include tracing the journey of a mailpiece as it makes its way from printer to mailbox. (Or tracking the Delorean as it travels back in time?)
Bring your own SQL
In a previous hackathon, data analyst Chris Migirdic built an IDE where you present your own time series data/problem, and a layer could be provided over it to make time-series predictions. This iteration was quickly dubbed “BYOSQL,” or Bring Your Own SQL, by participants as he explained one could write any SQL query against Lob’s data warehouse, and you'd be able to fit a time series model over that. The most obvious application for Lob is evaluating mailpiece data. Another improvement over the previous version was performance; much of this was attributed to a shift to deployment in Nomad. Finally, another benefit of the program is to pick up what kinds of queries people are running and get an idea of what prediction problems our developers are trying to solve.
SFTP to the future
Solutions Engineer Sam Melero has noticed a trend amongst our enterprise customers: their technical teams are the likely beneficiaries of investment in the latest tools and technology (the future), while the marketing teams may not (and are stuck in the past). For example, in the world of marketing automation, secure file transfer protocol (SFTP) is still a fairly common communication method. This may seem like a mismatch with an API-first company like Lob, but we are able to successfully onboard these customers as-is, and we support them as they transition.
Anything we can do to support our customers as they transition to API benefits both parties. The goal of this project was to modernize STFP to work more efficiently with Lob’s existing architecture/processes. AWS technology stack was utilized to achieve these improvements including CloudFormation template, Secrets Manager, and Identity and Access Management (IAM)—with lots of Lambda functions in the mix. In short, this allows the user to log in more easily and securely to upload their creative to Lob, and have the same experience as an API user. (For example, API requests will be visible in the dashboard along with the appropriate analytics.)
Automating alerts
Ken Pflum, an engineer who works with our print partners, has been frustrated by the number of alerts coming in as many of them require manual intervention—and he had a suspicion that wasn’t always necessary. Thus the birth of the Automat, which is in essence, a webhook handler and runbook automator (tools/technology utilized included Datadog and open-source Redis). Upon alert, a webhook gets sent into the service, which can then trigger a designated workflow. For the proof of concept, the example was to automatically send an email to the partner from the alert. A small step, but definitely one in the right direction since that is a manual process right now. (Besides, it could be dangerous to move too fast. “If my calculations are correct, when this baby hits 88 miles per hour, you're gonna see some serious sh*t.”)
It’s pretty amazing what can be done with a little focus time. After all, per Marty McFly, “If you put your mind to it, you can accomplish anything!” And speaking of Marty, I’ll let him take us out: