a manifesto about application observation
1,620,000,000,000 MPH: The New Speed of Business
Let's get right to the bottom line. We've let developers down. For all of the talk about them being kingmakers, we sure haven't treated developers like kings. In fact, we have treated them pretty badly.
In today's business climate, we expect developers to continually crank out wildly complex and ever more powerful applications without giving them the tools they need to see what is happening to those apps once deployed. It's 2019, and we think that no developer or business owner should be in the dark when it comes to knowing what's going on in their app. Unfortunately, we don't see the right solution on the horizon, and the current situation is only getting worse.
App developers have no idea what is happening in their application because they rely on historical information; looking at data that is months, weeks or days old, waiting for data that isn't delivered to them directly or trying to use data that isn't pertinent to the problem they are trying to solve. Developers are deploying apps blindly and then left to sort out any issues in a chasm with no chance of climbing out.
This mistreatment needs to end.
It's time for developers to stop using app data to find out "What happened?" and start using it to ask, "What is … ?" What is happening right now?" Not, what happened yesterday, what happened last week, or what happened a month ago. What. Is. Happening. Right. Now.
[bctt tweet="It's time for developers to stop using app data to find out What happened? and start using it to ask, What is...? What is happening right now? Not, what happened yesterday, last week, or a month ago. Right now."]
But here's the nut ... we have a data paradox. The applications we're asking developers to build now generate more data than they ever have. They will create more data next year, and the next, and the following year after that. The challenge can only get worse. We're no longer in a world of managing data streams ... we are talking about managing massive flows; data rivers at full crest after a thousand-year rain event. It's overwhelming developers and pushing them out to sea.
But wait, having access to lots of data is a good thing, right?
You might think that with all this new information availability developers and business owners would be awash in fine grain, valuable knowledge about how their apps are performing.
You might think that it would be easy to understand app and device performance across environments, across devices, and regardless of location.
You might think business would have better insight into the user experience and how customers are using their products.
But you'd be thinking wrong.
There's so much data available that much of it is useless by the time the business can aggregate, access, and digest it. Large amounts of data are entirely ignored. Given currently available tools, developers are forced to rely on past data to see what has already happened without visibility into what is happening - right now. All this new data isn't helping - not even with the most basic business questions. Businesses don't know when their customers are having a poor experience with their app because they only find out about problems after someone complains or the app crashes. And we all know that by the time you hear about a problem, it can be too little, too late.
Let's get real.
The amount of data driven by southeast Asia's Uber competitor, GrabTaxi, is a prime example of the challenges modern businesses face. GrabTaxi processes 450,000 messages per second through their app. Chew on that. That's the equivalent of 1.62 trillion messages an hour. That's the new speed of data.
At 1.62 trillion MPH, minutes-old information is useless to a business. For a company like GrabTaxi, there is no time for "What happened?" Businesses that don't know "What is...?" won't keep up. And it's not like applications are suddenly going to start generating fewer data. That ship sailed long ago. The developers' data paradox will only get worse unless someone does something to fix it.
We think Hrbr is that someone, but to be honest, we don't care who fixes it, just that someone makes it easier for developers to know what is happening in their app.
Sounds simple enough - but what does a fix look like?
The Hrbr team believes there are three critical elements to the fix:
Fix #1: Cloud-based delivery that matches development reality
Today's businesses and applications are cloud-based. We believe cloud-native tools and cloud-based infrastructure is the only approach that makes sense in the current environment of microservice architectures, distributed and asynchronous hybrid applications with many endpoints, and data volumes that continue to grow exponentially. Cloud native development needs tools that can capture information regardless of location. No open source installations, no tools trapped on a server that needs to be managed by an administrator. Cloud-based, cloud-native tools.
Fix #2: Developer-defined "pace of measurement" and "points of curiosity"
Every business and every developer has information needs that are unique to their mission. It's our view that developers lack the tools they need to understand what is happening in their applications because existing tools aren't optimized to ask the right questions. Legacy tools are pre-configured with what you can watch or query, not want you WANT TO KNOW. Traditional monitoring tools only look at servers and infrastructure and were created for server-based deployments - not the cloud. Logging is wholly focused on what happened in the past. "Sampling" implies grabbing snippets of data - over time - but who defines the time increment and what is being grabbed? In the old way of doing things, the only perspective is what happened? It's time to let developers define the intervals and the insight that makes sense to them, their app, and their business so they can arrive at "What is ...?"
Fix #3: Lightweight, quickly deployed, developer-centric tools ... priced right
Bless your heart, IT - you tried. You really, really did. And we love you for it. But the application monitoring, logging, and management tools built for IT needs are not optimized for developers and today's applications. Developers need lightweight, flexible, open source tools to see what's going on in their widely distributed, hybrid apps and devices. And let's not forget that the brand name tools that "everyone is using" add massive overhead, and are incredibly expensive, to boot. We can't expect developers to understand and react to hundreds of thousands of message events a second with tools built to deliver 24-hour old data and pricing models based on servers and seats.
The fix is in.
But how does knowing "What is ...?" fix the problem and enable developers to do things differently?
Imagine, if you will you're moving down the information superhighway at 1.62 trillion MPH, windows down, and the tunes cranked up. Would you rather be looking in the rearview mirror with a white knuckle death grip on the wheel hoping you don't slam into something and come to a sudden unplanned stop? OR do you want to keep the pedal glued to the floor with your eyes on the road ahead looking through a modern heads-up display confident you'll be able to see and respond to what's coming at you?
Of course, everyone wants the second alternative, and that's how we define application observation.
Hrbr enables you to embed developer-centric application observation so you can know in real-time what is happening in any app, on any device, on any platform, regardless of where it's deployed.
[bctt tweet="Hrbr enables you to embed developer-centric application observation so you can know in real-time what is happening in any app, on any device, on any platform, regardless of where it's deployed."]
And since no one has time to waste – because 1.62 trillion MPH is a bit zippy – Hrbr is optimized for quick deployment. In just a few minutes developers can embed Hrbr Beacons to begin capturing the data they want and automatically send lightweight JSON messages to Hrbr Stream Svcs - our ultra high-volume cloud-based stream services layer. Hrbr Foghorns and Tugs deploy just as quickly to enable custom alerts and actions based on Beacon message content.
This combination of lightweight code, quick deployment, and high-volume message processing means near-real-time app observation and action. And remember how we mentioned that old tools are expensive because they weren't built and priced for how developers work today? That's why every Hrbr account includes 5 million free messages per month, and then pricing is usage-based. Developers can finally define – and pay for – just the information that's valuable to their business instead of excessive, annual license fees for solutions which address only a portion of their needs.
Using Hrbr developers can deploy Beacons to capture what they want to know for as long as they want to know it and pay for it for as long as you need it. We think that's a pretty novel idea, though it shouldn't have to be.
Tools to navigate a "What is ...?" world
It's now the age of "What is...?" and it's time to get moving. The real-time world everyone envisioned is here and rushing past us at a rate of 450,000 messages a second. Hrbr wants to equip the architects and builders of that world with the tools they desperately need to navigate successfully.