“数字+”与之江统计讲坛（第77讲）1月3日伦敦政治经济学院Chengchun Shi来中心讲座预告

报告题目：ARMA-design: Optimal Design for A/B Testing in Two-sided Marketplances

报告时间：2025年1月3日 (周五)  15：00

报告地点：综合楼644会议室

报告人：Chengchun Shi

报告人简介：

Chengchun Shi is an Associate Professor at London School of Economics and Political Science. He is serving as the associate editors of JRSSB, JASA (TM) and JASA (CS). His research focuses on developing statistical learning methods in reinforcement learning, with applications to healthcare, ridesharing, video-sharing and neuroimaging. He was the recipient of the Royal Statistical Society Research Prize in 2021 and IMS Tweedie Award in 2024.

报告摘要：

Online experiments are frequently employed in many technological companies to evaluate the performance of a newly developed policy, product, or treatment relative to a baseline control. In many applications, the experimental units receive a sequence of treatments over time. To handle these time-dependent settings, existing A/B testing solutions typically assume a fully observable experimental environment that satisfies the Markov condition. However, this assumption often does not hold in practice.
This paper studies the optimal design for A/B testing in partially observable online experiments. We introduce a controlled (vector) autoregressive moving average model to capture partial observability. We introduce a small signal asymptotic framework to simplify the calculation of asymptotic mean squared errors of average treatment effect estimators under various designs. We develop two algorithms to estimate the optimal design: one utilizing constrained optimization and the other employing reinforcement learning. We demonstrate the superior performance of our designs using two dispatch simulators that realistically mimic the behaviors of drivers and passengers to create virtual environments, along with two real datasets from a ride-sharing company. A Python implementation of our proposal is available at https://github.com/datake/ARMADesign.