English | MP4 | AVC 1280×720 | AAC 44KHz 2ch | 40 Lessons (8h 02m) | 918 MB
Optimize the performance of your systems with practical experiments used by engineers in the world’s most competitive industries.
In Experimentation for Engineers: From A/B testing to Bayesian optimization you will learn how to:
- Design, run, and analyze an A/B test
- Break the “feedback loops” caused by periodic retraining of ML models
- Increase experimentation rate with multi-armed bandits
- Tune multiple parameters experimentally with Bayesian optimization
- Clearly define business metrics used for decision-making
- Identify and avoid the common pitfalls of experimentation
Experimentation for Engineers: From A/B testing to Bayesian optimization is a toolbox of techniques for evaluating new features and fine-tuning parameters. You’ll start with a deep dive into methods like A/B testing, and then graduate to advanced techniques used to measure performance in industries such as finance and social media. Learn how to evaluate the changes you make to your system and ensure that your testing doesn’t undermine revenue or other business metrics. By the time you’re done, you’ll be able to seamlessly deploy experiments in production while avoiding common pitfalls.
Does my software really work? Did my changes make things better or worse? Should I trade features for performance? Experimentation is the only way to answer questions like these. This unique book reveals sophisticated experimentation practices developed and proven in the world’s most competitive industries that will help you enhance machine learning systems, software applications, and quantitative trading solutions.
Experimentation for Engineers: From A/B testing to Bayesian optimization delivers a toolbox of processes for optimizing software systems. You’ll start by learning the limits of A/B testing, and then graduate to advanced experimentation strategies that take advantage of machine learning and probabilistic methods. The skills you’ll master in this practical guide will help you minimize the costs of experimentation and quickly reveal which approaches and features deliver the best business results.
search inside this book
What’s inside
- Design, run, and analyze an A/B test
- Break the “feedback loops” caused by periodic retraining of ML models
- Increase experimentation rate with multi-armed bandits
- Tune multiple parameters experimentally with Bayesian optimization
Table of Contents
1 Optimizing systems by experiment
2 Measuring by experiment
3 Why are experiments necessary
4 Summary
5 AB testing Evaluating a modification to your system
6 Take a precise measurement
7 Run an AB test
8 Summary
9 Multi-armed bandits Maximizing business metrics while experimenting
10 Evaluating multiple system changes simultaneously
11 Thompson sampling A more efficient MAB algorithm
12 Summary
13 Response surface methodology Optimizing continuous parameters
14 Optimizing two or more continuous parameters
15 Summary
16 Contextual bandits Making targeted decisions
17 Explore actions with epsilon-greedy
18 Explore parameters with Thompson sampling
19 Validate the contextual bandit
20 Summary
21 Bayesian optimization Automating experimental optimization
22 Model the response surface with Gaussian process regression
23 Optimize over an acquisition function
24 Optimize all seven compiler parameters
25 Summary
26 Managing business metrics
27 Define business metrics
28 Trade off multiple business metrics
29 Summary
30 Practical considerations
31 Don’t stop early
32 Control family-wise error
33 Be aware of common biases
34 Replicate to validate results
35 Wrapping up
36 Summary
37 Linear regression and the normal equations
38 Multivariate linear regression
39 One factor at a time
40 Gaussian process regression
Resolve the captcha to access the links!