Absolutely. Kanban is not a straight jacket limiting you to only one way of doing things.
Kanban is a visual representation of the actual workflow. In hardware it can be pretty obvious as you're moving around a physical object. If a car fails quality inspection, it doesn't sit in the quality bay waiting for an engineer to come to it. It gets sent back to where the work needs to happen.
If QA finds a bunch of issues with a completed piece of software, they send it back to development. If it stays in QA, it limits QA's work in progress limits. Only QA isn't working on it anymore.
And moving something even as far back as To Do is a recognition that you can't work on it right now. Maybe a dependency popped up and you have to wait a month for that to be complete.
Now I do recommend a "Waiting" column on your task flow. If you are waiting for an outside dependency, which is within a reasonable time, then toss it in waiting. It shows it is being worked on, but not actively. Things in waiting would not count again the WIP limit for "Doing". I still put a WIP on "Waiting" if you exceed that WIP, then you start raising risks.