I'm testing out Kanban on my research team and I think we're okay for now. We've visualized our workflow and set limits to the tasks. I'm using Kanban in Action for reference. I've set an initial WIP limit of 8 tasks for now (2 per member of my team). There are times when it works and times when it doesn't. My question is, how often and when do I change the limit? Can I do it in the middle or the week? or is it preferable to do it during retrospectives?
There is no need to change WIP limit to match the actual capacity. You can monitor its changes and bring that information to a retrospective meeting and see what is going on.
It is fine to have lower WIP than the limit; it means that you have more resources available than needed. When the same column exceeds the WIP means that there are cases when the available resources cannot handle the input; either the work items arrives too fast - previous phase is faster - or leave too slow.
Record - camera or screen shot - a couple of cases when the current number of work items exceeds the limit, and examine the scenarios during the retrospective meeting. With this approach you'll learn something about your own process.
In case the work is very often above the limit, then increase the limit.
It is ok to change these limits anytime, but people should definitly agree. It seems to make most sense to do it when most people are standing in front of the board having some sort of meeting or discussion like a daily standup, but middle of week or retrospective are fine too. My golden rule is a WIP limit that never gets hit should be lowered, and one that gets hit all the time should be raised.
You are supposed to hit the WIP limit! If you hit the WIP limit it means it is working!
2 per person is a good place to start. Generally you will start a team somewhere between 1-3 items per person on the team as a rough guidance. It's better to actually observe the team and set the limits at the lower end of what they naturally do. I can't stress how setting WIP limits too low on a transitioning team is a great way to fail a kanban transformation.
As for when to change the WIP limits there are 2 common scenarios:
If the team exceeds their WIP limit, DO NOT have the knee jerk response of changing the limit. Exceeding WIP 1-2 times a week during the first few months of transition is completely healthy and expected. Its also a great opportunity. The whole point of a WIP limit is to drive discussion of what can be improved in the process so that the team doesn't keep hitting its WIP limit. That said, a team that is daily hitting their WIP limit and can't seem to make any process changes to address it, your limit may be too low.
Now, when you notice the team is stable (for a few weeks to months) and consistently staying at or below their current WIP limit, its a good time to start having discussions about tightening the belt and lowering the limit further. Again this should lead to discussions about how to eliminate waste in the current workflow.
Kanban by David Anderson is a great resource that gets much deeper into the weeds on this topic.
PS if you're doing Kanban in a sprint context its not really Kanban (there are no iterations or sprints in pure Kanban). You're probably doing something more akin to Scrumban. Nothing wrong this as long as you understand why you are mixing and matching tools from the different frameworks.
You shouldn't be changing it mid-week or mid-sprint, and it shouldn't be a constant thing that you change - you're aiming to find the right WIP limit and then stick with it. There will be tension around the limit, that's the idea of having restrictive processes in the first place - to enforce practice that leads to good flow.
A retro would be a good time to discuss and justify the change.
eg: Is the current limit being too restrictive of flow? Is the current limit allowing for too many things to happen and lots of things are ending a sprint incomplete?
There are times when it works and times when it doesn't.
It would be worth explaining this more. What do you mean by this?