Archive | WAW Recaps

February 2020 Recap – Factor Analysis with Ahmad Ahmad

Our February meetup was a fun and informative session on factor analysis with Ahmad Ahmad downtown at Hopewell. Thanks to everyone that showed up even though the weather was kind of awful! Ahmad first gave us an introduction into the concept of factor analysis and when it might be helpful, then in good cbuswaw style proceeded to show us some real data and analysis, throughout fielding some solid questions from the crowd.

Some data is easily reducible to a smaller set of groups. For example, during Ahmad’s talk there was a significant amount of water droplets falling from the sky. These droplets could be very easily reduced into different types: rain, snow, freezing rain, and sleet. This is a case where we definitely don’t need factor analysis, because these are directly observed variables, i.e. we know what precipitation is what without using any statistical methods. Nobody would ever ask an analyst to quantify if it was snowing or not, but they would ask us to figure out from a bunch of different web stats why their users aren’t converting. How do we boil those dozens of dimensions down to groups of a few useful ones that share some kind of common underlying dimension that was not directly observed? And then what happens with even less obviously reducible data? We need statistical tools! Ahmad first walked us through a couple of thought exercises on this kind of dimension reduction to see how this might function from a high level.

Ahmad knew to bring data though, so he brought an analysis that he did based upon Boeing employee survey data where he attempts to turn 30-something different questions into a useful number of higher level groups, in this case 4. He did the analysis in SPSS, but he brought the results into Excel for presentation and has been kind enough to provide that Excel file as well as his slides:


Included in his slides are links to more info on factor analysis using the following tools:
SPSS (with Varimax Rotation)
Excel (with XLSTAT add-on)
Python (with Factor Analyzer package)
R

Those of us that also saw Dr. Michael Levin talk on cluster analysis last September may also want to revisit his talk to remind ourselves of the differences between these two very similar topics that go hand-in-hand but have different uses. Factor analysis is about discovering these underlying groups (like finding the survey questions that could be represented into an underlying factor of “career stage” — age, expected retirement date, and less obvious dimensions like perception of selling potential). Cluster analysis is about sorting the surveyed people into groups by using the existing dimensions, such that group members are more self-similar than people in another group.

Please join us next month when we’ll be back at Rev1 learning about service design with Monica and Anthony Weiler.

January 2020 Recap – Analytics Consulting with Elizabeth Eckels

We kicked off the new year at Otterbein University, we’ll be at Hopewell next month and then back our regular digs at Rev1. In a well-attended lecture hall Elizabeth Eckels, CEO & Founder of Bancroft Digital, presented an intro class into what the world of consulting is all about. Not about the tedious details of  how to setup an LLC or your corporate tax structure, but a lively conversation about analytics consulting practice itself, with real talk about the pluses and minuses.

The audience was fully engaged for this one, as basically everyone has either spent some time as a consultant or engaged with a consultant as a part of their job. There’s a ton to learn from both sides; learning how to most productively engage with your consultant is important too! Going out on your own in whatever the form can present a lot of risks and benefits, but Elizabeth walked us through both in a thoughtful way.

A few of her tips:

  • Don’t undervalue yourself.
  • Especially if you work at home, setting work boundaries can help keep you sane.
  • Frequent and thoughtful client communication is incredibly important. Put yourself in their shoes, but keep records to protect yourself too!
  • Nobody knows if you’re wearing pajama bottoms on a video conference, unless you have to stand up.

She has also kindly provided her slides! Check out slide #18 for a list of resources.

Welcome to the many new faces this time, hope to see you at future events as well! Also welcome back to one popular canine face, Elizabeth’s dog Cash.

Please join us next month downtown at Hopewell where Ahmad Ahmad will present on factor analysis.

November 2019 Recap – Analytics in Context

Our final regular meetup of the year was a fun strategy session with Jen Heider about managing and understanding analytics in context.

What exactly does “in context” mean? Jen explained it’s all about fitting our analytics projects into the larger context of the business as a whole. Look, we all agree that numbers and algorithms are great just for their own sake (though maybe not everyone else does), but letting them lead us is just a backwards way to go about an analytics project. Why we are doing an analysis, and what kind of decisions we might facilitate with that analysis is way more important that if we used naive bayes or random forest. Anyways, we probably should have just started with a basic regression.

Jen was very up-front that much of what she was presenting was the result of mistakes that she had made. We think she can join the club of, well, everyone in that, but she joins a much more elite club that owns those mistakes and learns from them. Can a model using the titanic disaster dataset help predict someone’s likelihood to throw themselves under the bus? We don’t think so, but maybe Jen can figure this out and get back to us.

We learned that an analytics practice shouldn’t be isolated away from the rest of the organization, where other departments submit their questions along with an offering to Apollo and wait for an answer that may or may not be understandable (or correct!). Integration of analytics into the context of the rest of the business allow for very important things including:

  • Choosing the right analytics project based upon scope and effect (but maybe take it easy with those effort vs. impact matrices).
  • Defining metrics across the organization so that people agree what they mean and how they are used.
  • Communication to know if the results are actually understood!

This allows partnerships to build over time in a really productive way. Less duplication of efforts, less reporting that is never even looked at, and ultimately (we hope) better decisions made.

Please join us next month for our annual holiday meetup at North High Brewing. No speakers, just fun & socialization!

October 2019 Recap – Product Analytics

Our October meetup featured Martijn Scheijbeler from RVShare talking about what “Product Analytics” means and how it’s different than Web Analytics. Since there’s no days of the week that start with “P” we aren’t going to be renaming “Web Analytics Wednesday” to “Product Analytics Pieday” (although we are open to the idea of a pie-oriented day), but that shouldn’t stop us from thinking more deeply about how to bring a more product-oriented perspective into our measurement practices.

Traditional web analytics works based upon sessions & pageviews. Having run Web Analytics Wednesdays for the last 11 years we feel  pretty confident in this assessment. This paradigm works well for aggregate stats about our sites, but when trying do user-based analysis of a whole product from a holistic perspective this method can really start to break down. Adding event tracking helps, but in traditional tools like Google Analytics events lack the deeper context to answer a lot of the questions about how these different events fit together into one user experience.

So what do we do? Once we’ve hit the goal & custom dimension limit in GA do we just need to start rolling our own complicated in-house analytics tools??

Luckily, Martijn just happened to bring along with him someone eminently qualified to show us what might be next! (Ok, they are married so “just happened” may be an overstatement). Our surprise guest was Krista Seiden, Founder & Principal Consultant from KS Digital, and well-known as a former Google Analytics Advocate at Google.

While at Google Krista most recently worked on the newest version and future direction of GA, App + Web properties. Krista’s blog is one of the best sources of information on this new version of GA, which takes an events-first approach and addresses many of the challenges Martijn had been talking about. Of course, A+W is still in Beta and even in full release is not going to solve all of the issues  discussed, but it is far more than just a way to consolidate mobile app and website stats. Both of our speakers also mentioned a number of other tools working to take our analytics up to this next level (Segment, Snowplow, Heap, Mixpanel, etc.) including many that do already take the events-first approach that A+W has adopted.

 

Martijn has also kindly provided us with his slides:


Please join us at Rev1 again next month when Elizabeth Eckels will talk about the world of contracting & consulting in Columbus.

Other Upcoming Events & Conferences:

Oct 23-24, DAA One Conference, Chicago
Oct 24, Market Research Exchange Fall Conference
Nov 6-7, Business Agility Conference Midwest

 

September 2019 Recap – Cluster Analysis with Dr. Michael Levin

Our September meetup featured a strong turnout for the always popular Dr. Michael Levin from Otterbein University speaking about cluster analysis. We’ve checked the records and this was Dr. Levin’s 4th time presenting! An impressive feat which puts him close to the free tote bag for members of the five-timers club. Considering the quality of the content and the great questions it engendered we better start designing that tote bag!

So what exactly is cluster analysis?? “K-means cluster analysis” — it sounds kind of esoteric and difficult, but Dr. Levin showed both how crucial this kind of analysis is and as the ease with which it can be implemented. We might have 10,000 different individual customers, but if we want to actually analyze and then take actions upon those customers we really need to split them up into a manageable number of groups.

Don’t forget that groups of everyone combined or everyone one at a time are still groups, just not very useful ones! Useful groups are the smallest number of groups we can have that split up our set clustered by the dimensions that we are interested in.

Dr. Levin walked us through an example of this kind of grouping with real world data and was brave enough to actually bring up Excel to do a live coding example. Typically that’s a good way to make sure everything explodes, but the only breakage was a few brief projector outages.

He was also kind enough to share both his slides and his Excel templates! The four cluster approach comes from Wayne Winston’s book “Marketing Analytics: Data-Driven Techniques with Microsoft Excel“.

Excel Templates:

Three Cluster Solution Template
Four Cluster Solution Template
Five Cluster Solution Template


 

This kind of analysis can of course also be done in your statistics package / programming language of choice. We will now provide a couple of links on how it can be done in R or Python to satisfy our toolset “fairness doctrine” requirements, as mandated by the cbuswaw bylaws. As a bonus these also shows just how simple excel can make it!

R
http://markedmondson.me/intro-to-machine-learning-with-web-analytics-random-forests-and-k-means
https://uc-r.github.io/kmeans_clustering

Python
https://towardsdatascience.com/an-introduction-to-clustering-algorithms-in-python-123438574097

Please join us next month when Martijn Scheijbeler from RV Share will discuss product analytics!

August 2019 Recap – Testing Strategy with Melanie Bowles

Our August meetup was an excellent session on A/B testing strategy with Melanie Bowles from InfoTrust. To go along with the theme of A/B testing we also stepped up our door prizes and offered the crowd multiple variants of door prizes: including AirPods and wine from campaign tracking service Claravine.

A vitally important but frequently overlooked part of doing A/B testing is the structure behind the testing. You might ask, “how could we possibly need a team of people, launch checklists, test priority queues, and all this other stuff if testing is as simple as ‘just one line of JavaScript on your site'”? Well, it turns out marketing is not always 100% true (shocking news!!) — and while implementing the testing tracking snippet itself might be pretty easy, there are many other steps in the process that aren’t so trivial.

Melanie did talk a bit about how one might evaluate different testing tools, but she was smartly tool-agnostic in her presentation. While there are many great discussions to be had about the different tools and the math behind them, without a good strategy on items like how to generate testing ideas and prioritize running those tests you could have the best tool in the world and your testing practice could stall out and go nowhere.

A key part of the testing process is consistency and replicability. This requires a good strategy thought about ahead of time! Anyone who has run multiple A/B tests will know that actually making decisions from your test outcomes can be hard. Simply running a test without deciding before-hand what your success conditions might be is very tempting, but it’s rarely the case (especially with a mature product) that the results will speak for themselves in a vacuum — and then what do you do?

Melanie recommended using templates to make sure your process is consistent and sustainable, and was kind enough to provide her slides including some example templates!

Please join us again next month back at Rev1!
 

July 2019 Recap – Presenting Results to Inspire Action

For our July event we had a great turn-out to see Valerie Kroll from Search Discovery teach us about effective presentations. As part of a Search Discovery caravan down from Cleveland for the evening, Valerie presented a consistent strategy on getting the attention of stakeholders to drive action from test results.

The context of this strategy was A/B testing, but the larger points on presentation were totally relevant for any kind of presentation. No matter what kind of results you’re showing we were reminded:

  • To focus on the key results from the perspective of your audience.
  • Even a “failed” project can be an opportunity to learn important things.
  • It’s possible to boil down the results even more than you might think! A two slide presentation can be enough, and after all you are the real conveyor of content, not the slides.

Maybe it’s new to you, or maybe you think you’ve heard this stuff before — but focusing your presentation down to the simplest and clearest version of the results is one of the consistently most difficult (and important) part of our jobs.

A consistent methodology for both the creation of a testing hypothesis and the presentation of results before you start actually running anything can be crucial, but is also a lot of work. Valerie showed us a very useful template for presenting results, and has been kind enough to share her hard work by making the PowerPoint templates available at the Search Discovery site here.

Valerie has also made here slides available!


Please join us next month at Rev1 again for more on testing when Melanie Bowles from InfoTrust will present on building an experimentation strategy.

June 2019 Recap – Mobile App Analytics

For our June event, Mai Alowaish from Blast Analytics and Marketing made the trip down I-71 from Cleveland/Akron to share an information-packed presentation on the many facets of mobile app analytics.

Her presentation covered:

  • An overview of what mobile app analytics is (and how it differs from mobile site analytics and hybrid app analytics)
  • The different underlying types of app analytics: marketing analytics (downloads, shares, deep linking performance, etc.), performance analytics / app health (crashes, errors, latency, etc.), and in-app analytics / product analytics (funnel behavior, personas and demographics, drop-off points, etc.)
  • The myriad different platforms for app analytics — which type(s) of app analytics they cover, as well as what their interfaces look like and enable
  • The different considerations when it comes to how to implement app analytics: to TMS or not to TMS? API hubs? CDPs?
  • How to actually go about planning what to track (see the speech bubble below for a key to that!)

The presentation is available for detailed perusal here.

We had a full house of engaged attendees!

And, as we’ve been doing all year, we had Columbus Web Analytics Wednesday T-shirts as a door prize drawing! One of the lucky winners was actually in town from the Bay Area, so we pretty much assume that “cbuswaw” will be assumed to be a hot new startup inside of a week, and we’ll be fending off venture capital funding offers:

If you’d like to experience the presentation almost as though you were there:

  1. Load up a plate with a few slices of pizza
  2. Get yourself a tasty beverage
  3. Watch the video below that Mai was kind enough to record with her slides and her voiceover!

We’ll be continuing our streak of fantastic content from out-of-town speakers next month when Valerie Kroll joins us to share her tips for presenting results that inspire action. We hope to see you there!

May 2019 Recap – The Path(s) from Data Analyst to Data Scientist

For our May event, we cast our speaker net out-of-state and convinced Jim Gianoglio from Bounteous to make the trip from Pittsburgh to share his experience and his thoughts on the myriad paths that exist for perambulation from “analyst” to “data scientist.” Or, as Jim subtitled his talk: “the transfiguration from reporting squirrel to unicorn:”

It was a packed house for the event, as Jim walked through his various explorations of options for advancing his analytics skills into the world of data science, which he boiled down to three options:

  1. Entering a formal degree program (online or offline)
  2. Relying on the various online courses and content that are available for free or a nominal fee
  3. Attending a bootcamp.

Jim initially dabbled in online courses, but, ultimately, went for a formal degree through Carnegie Mellon. The pros of that approach:

  1. The cost and face-to-face schedule meant that, even as the going got tough, bailing wasn’t really an option.
  2. The in-person interactions with professors and students made for productive collaboration and deeper learning (…including on the subject of — wait for it — deep learning, presumably </editorial license>).
  3. The networking benefits — in a traditional sense, this would mean that Jim was set up to hop to another role following the program, but, in this case, it meant that two of his fellow students got hired by Bounteous!
  4. The cachet of having a Master’s degree from a school like Carnegie Mellon — that’s good for the resume!

Of course, there were also downsides:

  1. It was an intensive and exhausting two years, as Jim continued to work full-time throughout the program, while also having a wife and three young children.
  2. It wasn’t cheap. Jim did the math as to how/when he would expect a return on his investment, and it made sense.
  3. There were still some “dud” professors, which can also happen in the online world, but, when you find yourself calculating a “cost per hour” during a lecture and getting a little steamed, that can be disheartening.

While Jim opted for the in-person, formal degree program, he also discussed — and provided a number of resources — for other options (some of which he availed himself of both before and after his formal coursework):

  • Online degree programs from accredited universities
  • Open courseware and content — use resources like the Open Source Data Science Masters to put together your own curriculum!
  • Bootcamps — although Jim warned that there is an explosion of these being offered, so the quality varies wildly, and bootcamps can make unrealistic claims (“Become a data scientist in just 14 weeks with our bootcamp!”)

Ultimately, there are an overwhelming number of options, which can be intimidating, but it also means that analysts can do some research and introspection and then figure out what is the best option for them!

Ultimately, with a little bit of statistics, some Python, and a little bit of R, you, too, can catch yourself speaking like a data scientist!

Jim shared his slides (with notes) here if you missed the event or attended and would like to reference the material. A smattering of resources he referenced and recommended are:

Join us in June for a discussion of mobile app analytics as Mai Alowaish from Blast Analytics & Marketing shares tips and best practices for mobile app analytics!

April 2019 Recap – The Future of Driving

For our April event we had husband and wife duo Kevin Boehm and Sharon Santino lay out the current state of autonomous driving as well as where we are headed down the road (ok, we promise no more car puns).

If you read some of the tech press or Elon Musk’s twitter feed, then you might think that we’re only months away from just laying back and letting our smart cars do all the work, but that’s not quite the case.

Sharon and Kevin brought our flying smart car dreams back down to earth a bit by explaining many of the challenges involved, but they also showed some of how revolutionary this technology will be when it does eventually fully arrive.

As usual, most of the engineering problems are related to people and their unpredictable behavior. While the cars may be getting smarter and smarter, people will remain people.

They also laid out how it’s not an all-or-nothing process, but much more of a continuum — and while fleets of cars at scale with no steering wheels at all may still be pretty far away, there’s also lots of this technology already out there.

Please join us next month when we’ll have Jim Gianoglio from Bounteous talk about the path from Data Analyst to Data Scientist!

 

As a bonus, check out the cool time-lapse that Sharon and Kevin made!