What is endurance? How are you going to improve it? Somethings to think about...
1. Ride more. But what type of ride is it? When you say your going out for some endurance base miles at the weekend, are you? Unless you have a group of riders with the same objective and same level of fitness, your group ride could be far ...from an endurance ride. You could have such a range of fitness levels that 1 or 2 riders maybe doing what you would call a Z1-Z2 endurance base ride, but the rest could be having a threshold/interval session. One persons Z1-Z2 could be anothers threshold. Unless it's a conrolled ride within your Z1-Z2 range, it's not really going to be base miles.
2. Improve your threshold. Together with some proper base miles, this should improve your endurance.
3. Learn to to burn fuel more efficently. Is this improved by riding more or having correct nutrition or both? Are you a carb or fat burning machine, which of these is going to give you more endurance, which does your body rely on?
4. Ride your bike more efficently. Are you getting the most out of your bike, ie bike position? Are you getting 100% out of the effort you put in?
Improving your endurance is not just about one of the above, but a good mix of all of them.
There are alot of questions here, but they're to make you think about how you can improve your endurance.
There is alot more to improving your endurance than just getting the miles in!