1

I would like to know if there are any suitable estimation approaches for ETL (Extract, Transform, Load) projects. In my case I will be using Logstash for an elastic stack.

Since the amount of work (coding, testing, etc.) heavily depends on volume and variation of data, the labor cost is difficult to predict.

I appreciate if anyone can share one's ideas for how to estimate such projects.

0

Here are some ways to mitigate potential increase in scope of work

You said:

Since the amount of work (coding, testing, etc.) heavily depends on volume and variation of data, the labor cost is difficult to predict.

I suggest 3 options, in descending order of priority:

  1. Propose initial estimation work at about 5% of the cost of the project: When you complete this work, your deliverable will be a firm quote for the project.
  2. Ask for size (volume) and sample data from the client: The client can give you rough volume information such as number of records, size of the index, size of the data in disk...and so on. Also they can give you a very small extract from the data that helps you assess the variation. You can compare this size with other similar work you have done before. After seeing the variation in the sample, you can add a padding of some % based on your experience in other similar work.
  3. Give a quote stating your assumption of volume and variation: Tell them if the actual volume and variation are more, you will charge extra by providing proper data to support the justification for the extra charge.
  • Thank you for the reply. Your 3rd answer gave me a hint for the estimation. – Yu Watanabe Mar 4 '18 at 8:21

Your Answer

By clicking "Post Your Answer", you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.