The paper “Residual Q-learning: Offline and online policy customization without value” is accepted for NeurIPS 2023! Checkout the paper, code, and website.