Perhaps you've heard the analogy of an empty jar with big rocks, small stones, sand, and water. If you put in the water in the jar first, and then the sand, and then the small stones, by the time you get to the big rocks, they won't fit anymore. But if you use the same jar and put in the big rocks first, carefully arranging them, followed by the small stones, sand, and water, you could fit everything.
Our spiritual practice is like the second arrangement. If we direct our energy towards the most important thing-- our liberation-- by focusing our senses and our energy inwards instead of using it to attend to our cravings and attachments and aversions, our practice will be steady, our inner lives rich, and we will still have room to live our lives from the physical and external plane.
If we are interested in liberation, then let our practice be our big rock, and let us direct our energy towards it.