On one hand, software development involves a lot of non-programming activities like requirements gathering (including analysis), estimating, planning, testing, tracking work, change request management, and more.
On the other hand, our clients don't want to pay for these activities. When a non-programming activity shows up on their invoices, they usually get angry and complain to upper-management. Of course, they always win their cause. Therefore, we (the company) don't get paid.
The result of all this is that upper management is pressuring us (programmers and project managers) to not perform any of these non-programming activities. It generates all the problems that are commonly known to result for having a "code & fix" approach to software development.
Our poorly tested software is often released with bugs, and of course, clients don't want to pay for us fixing them. Moreover, we often release, just in time, software that don't correspond to what the client's wanted or needed because we never had the chance to invest the time to understand what the client needed. The release is therefore rejected by the client who then complains to upper managment. We then have to make all the requested changes for free.
I don't know how other companies solve this problem and I don't know where to look at to get the answer. Do they bill their clients for non-programming activities? More importantly, how do they get to finance these non-programming activities? Do they convince their clients to pay for these activities? Do they hide instead, unlike us, the real nature of the work they are doing? Do they charge so high for programming activities that it doesn't matter if non-programming activities don't generate any income? How?
I expect agile to be suggested as an answer, but please understand that I will not accept "you must do agile" as an answer, because it would not explain how agile would solve the problem. In a scrum team, for example, you perform a lot of non-programming activies and you still get paid for them. I cannot begin to imagine how our clients will ever accept to pay us for doing sprint retrospectives for example. If you want to answer "agile", please explain how agile solves the problem. For example, how do you finance all the non-programming activites happening during the scrum process?