I have two levels of to-do lists, and it's worked out pretty well for me. They're strictly for work though, I don't usually use them for my personal life:
Every morning I open an empty file in vim and create a list of things I intend to do that day. It's created completely from scratch without bringing anything over from the previous day, so this list doesn't end up growing forever with things I didn't get to, specifically to avoid that mental weight of "I have too much going on". I also remove them from the file as they're completed, instead of any sort of "mark as done", so it visibly gets smaller over the day and triggers an "I'm alllllmost done, let's finish these last things quickly" mindset instead of procrastinating as is easy when seeing a large list of things to do.
Then I use taskwarrior for anything I expect to last more than a day. I rely on its "urgency" column for prioritization, which I tweaked only once or twice in the 7 or so years I've been using it (mainly so team projects have increased urgency and special tags like "Hold" have it decreased). I rarely look at this one except when adding to it or when making the morning list, so I'll often come back to it and find a couple tasks I'd finished like a week or more ago and then spend a bit of time cleaning it up. So even though it can get big, I don't end up spending a lot of time optimizing it.
I think your method has worked well for me. I use Microsoft ToDo as a tool for it. It has a My Day list that automatically clears itself each day (by copying to Tasks). You can then re-add to your My Day list as needed.
Every morning I open an empty file in vim and create a list of things I intend to do that day. It's created completely from scratch without bringing anything over from the previous day, so this list doesn't end up growing forever with things I didn't get to, specifically to avoid that mental weight of "I have too much going on". I also remove them from the file as they're completed, instead of any sort of "mark as done", so it visibly gets smaller over the day and triggers an "I'm alllllmost done, let's finish these last things quickly" mindset instead of procrastinating as is easy when seeing a large list of things to do.
Then I use taskwarrior for anything I expect to last more than a day. I rely on its "urgency" column for prioritization, which I tweaked only once or twice in the 7 or so years I've been using it (mainly so team projects have increased urgency and special tags like "Hold" have it decreased). I rarely look at this one except when adding to it or when making the morning list, so I'll often come back to it and find a couple tasks I'd finished like a week or more ago and then spend a bit of time cleaning it up. So even though it can get big, I don't end up spending a lot of time optimizing it.