12

I have to do a quote for a contract work with a small company to do an AI task on images (classification, pattern recognition with specific conditions).

With "classic" programming tasks, you can know 99% in advance that, given enough time, you will succeed to build the product. But with an AI task, you can never promise in advance that "It will work with a 90% success rate!". Since there will be a part of research, you don't know in advance.

Question: How do you do a quote for AI R&D tasks, in which you don't know in advance that it will succeed?

  • If I charge only if the finished product works then I will have lost 1 month of work in the case the R&D is not successful.
  • If I charge even if the finished product is not working as expected (say 80% of success in the classification task), then the client will have lost money with a non-working product. It the client was a big company, I would do this. But here for a small company, I would feel bad to charge for something which is non working as expected for the client.

What are the most common ways to handle such projects?

4
  • Reminded me of this not very relevant xkcd: xkcd.com/1425 (read the title text, too).
    – wha7ever
    Dec 21 '21 at 13:34
  • 1
    All projects require risk management.
    – J...
    Dec 21 '21 at 14:30
  • Proper R&D tasks always have a risk of failing or you'd be doing a lot of D without R, isn't it? It's not only AI where this is valid. What does the company expect, an R&D project or an end solution? What would the contract state and does it correspond to the wish of the customer?
    – Mast
    Dec 22 '21 at 11:20
  • Closely related Q&A: pm.stackexchange.com/a/22607/4271, pm.stackexchange.com/a/6236/4271.
    – Todd A. Jacobs
    Dec 23 '21 at 17:56
20

No project has a 100% chance of success. So there's that.

For R&D, the project itself is not the product. It is research and development. It's a service-oriented project. Therefore, you would be reimbursed for the service, not what is produced or not produced. The output for a service-oriented project is a change in state. For R&D, that state is knowledge. At the end of an R&D project, the customer is smarter than before the project on whatever the customer was researching.

Therefore, you would create the type of reimbursement to be consistent with a service, which is typically cost plus fee on a time basis, i.e., T&M.

2
  • 5
    I mostly like this answer, but I can see being reimbursed for what is produced, if what is produced is some way to transfer knowledge. It could be documents and reports, mockups, prototypes. It really depends on how successful the outcome is. If the outcome is that there is a huge gap between the state of the art and what is necessary to make a product, then perhaps more on the documentation side. If what is necessary is close, you may end up with something closer to prototypes. There are often (but not always) deliverables to support the change in state.
    – Thomas Owens
    Dec 20 '21 at 14:34
  • @ThomasOwens, I agree 100% with your comment. Dec 20 '21 at 14:42
7

Good answers; I thought I might provide a different view.

I'd write my project as to test one (or more) hypothesis.

At the end of this project, we will verify or refute the hypothesis that it is possible to X (e.g. spawn a root shell using CVE-YY-xxxx).

I guess at a basic level, I'm challenging the validity of "then the client will have lost money with a non-working product. " - validating or refuting a hypothesis should both have value.

1
  • 1
    Rejecting or accepting the null hypothesis is a successful outcome! +1! Dec 20 '21 at 22:35
5

In addition to the good answers already given (T&M is the way to go) you should also put emphasis on customer expectation management. From the beginning, make it clear that you promise knowledge gain, not a finished product.

If the problem is one where it is known that solutions exist (for example because the big players in the field have them) and you just don't know how they do it, it's much more difficult. Your customer may expect you to build something they know can be built, but you may not be able to do that (as it requires R&D on your part, and you can't be sure that the R&D will lead to you being able to build the desired product). Your customer needs to be aware of those constraints before starting the project, as they may decide to look elsewhere, for example buying finished software, or contracting with someone else who already has experience in building such applications. Although that would mean you lose the contract, it might be better to be honest and spare the customer a disppointment.

3

I would expect any research project to be done on a daily or hourly rate. It's implicitly the nature of R&D that the work is what matters and the outcome is uncertain, so the work is what you should be paid for. A Kanban-style, continuous delivery approach makes sense for R&D: prioritise the items of value, inspect and adapt as you go.

2

Can you divide the project into phases and tasks? Can you provide a report, a review, prototype software and its code or some other deliverables of value for the customer? You may be able to mitigate the risks by doing that.

Are you supposed to be an expert with the domain knowledge? Or you are expected to carry out research on behalf of the customer? Then they can pay for your time, but also will want to have the ownership intellectual property and created associated with it.

You should also manage the customer's expectation and try to negotiate the best outcome for both parties.

2
  • Great answer. I am retired now, but when I was consulting I did just that. There would be a deliverable identified in the proposal for every task, so we could unambiguously show that we had delivered (it might be "source code in such-and-such a repo). Clearly identify risks in a spreadsheet ("risk watchlist") that you share and identify with the client on a regular basis; don't wait until the project is hit by a train--warn as soon as you an. Even progress reports should be included in the project as deliverables. Dec 21 '21 at 19:10
  • I think this is the most appropriate approach.
    – copper.hat
    Dec 22 '21 at 6:36
2

You can structure this as a T&M + Incentive contract

The goal is to avoid the client as well as the vendor being too unhappy in the event of a failed project.

To keep it simple, let us say that you need to spend 160 hours on this project and that your T&M charges are typically $100/hr. Let us also say that your resource cost is $60/hr and the rest is your mark-up.

You will quote a T&M rate of $60/hr and add an incentive payment of $6400 lumpsum if the outcome meets the success threshold.

If I charge only if the finished product works then I will have lost 1 month of work in the case the R&D is not successful.

You won't be too unhappy if you are not out-of-pocket by taking this project. Also, it is a good learning experience for you.

If I charge even if the finished product is not working as expected (say 80% of success in the classification task), then the client will have lost money with a non-working product.

The client won't be too unhappy if they paid the bare minimum out-of-pocket costs but not any mark-ups.

Most importantly, you will have a huge motivation to complete the project successfully and the client can see that too!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.