Our paper “Residual Q-learning: Offline and online policy customization without value” is accepted for NeurIPS 2023! Checkout our paper, code, and website.