Our paper “Residual Q-Learning: Offline and Online Policy Customization without Value” is accepted for NeurIPS 2023! Checkout our paper, code, and website.