This paper presents hierarchical-DQN (h-DQN), a framework to integrate hierarchical value functions, operating at different temporal scales, with intrinsically motivated deep reinforcement learning. Temporal abstraction is a challenging problem for any RL agent and  could enable better long-range planning behavior.