San Diego, CA, USA
dan@interfacedivision.com

Data Science Workbench

What to Do with Billions of Rows of Data

In the first few years after the mPulse launch we’d collected tens of billions of performance measurements from some of the biggest sites in the world.  We decided it was time to do some big data science with all of that data.

Data Science and Speed

Applying an old paradigm to a new problem.

The mPulse UI was written on the SOASTA CloudTest codebase which had been around since 2006, and as such was slow to update with new features.  By 2014, we’d identified a big need for the ability to do ad-hoc analysis, visualization, and big data science on the massive amount of data we’d collected for our customers.  Thus, the Data Science Workbench was created.

DSWB used a notebook paradigm, familiar to data scientists, but we made it approachable to performance engineers by writing our own easy to use function libraries for data analysis and visualization.  We heavily extended iPython/iJulia (now called Jupyter or JupyterHub) but built our own data science and visualization libraries in order to make it easy for non data scientists to use the platform.

Machine Learning for UX

Creating some unique intellectual property.

We went on to use the workbench platform to do accurate forecasting of UX and performance metrics using machine learning.  This was quite cutting edge in 2014, and is still relatively unheard of in the APM space.  We did big-data analysis across customers for industry benchmarking and comparison.  We also created the now-patented session path analysis algorithms and visualizations.  This allowed our customers to see the journeys customers took through their site with an angle on page performance at each step of the customer journey.

m

My contributions to this product:

Role: Vice President, Principal Product Designer (2014-2017)

  • Participated in the inception of the idea and created the groundwork from scratch, including but not limited to: requirements gathering, roadmapping, producing designs, creating the architecture, creating POCs, rapid prototyping, and ultimately delivering the working alpha version of the product
  • Hired and mentored a great data science team, almost all of which would follow us from SOASTA to conDati over time
  • Performed hands-on pre-sales work to win six figure deals in first quarter this product was released, at some of the biggest name brand companies in the world, ultimately driving revenue into the millions over the next two years
  • Our team was granted a patent for Session Path Analysis, a big data method and design for reconstructing journeys through a web application, with a special mind towards conversion and the performance of the pages at each step along the conversion path
  • Filed a patent for Test Plan Creation (pending), which was long considered to be a holy grail in testing.  This feature uses the clickstream data that mPulse collects from web sites in order to create a comprehensive performance testing plan.  This plan includes what functionality needs to be tested on a web site and what levels each area of the applications needs to be tested at to ensure stability during peak traffic periods like Cyber Monday
  • When SOASTA was acquired by Akamai, they told us it was because of our investment in data science