0

I work for a company that conducts modeling and analysis work on large 3D CAD and GIS datasets. For each project we work on, we can have hundreds of individual .las files (LiDAR point cloud data) that combine to cover the entire spatial area we wish to work on at one time. We find it is very difficult to track what data has been delivered and determine when complete spatial coverage for an area of interest is complete.

I have investigated various Product Lifecycle Management and Product Data Management softwares, but I found these are mostly meant for physically manufactured products and not necessarily suited for data products. I have also investigated various project management softwares, but these seem to track data about costs and schedules and do not track that raw data that is necessary for conducting analysis.

I realize this question is very broad and it is meant to be. I am unaware of any potential solutions to the data management problem and I am hoping others may be able to provide some insight.

To Recap: Our workflow requires several different types of data from various sources covering various spatial (geographic) extents. Are there software packages/systems out there for managing raw data collection and determining when the next step of a workflow is ready to proceed based on the types and amount of data currently available?

Thanks in advance for your help. I'm finding it difficult to describe what is need in a simple way as I have not found any examples of this problem elsewhere, so please ask questions to help clarify the issue.

1

At the level of project management you should not work with data, but only with metrics on top of them. Instead of trying to control the data - configure a mechanism of instant metrics collection.

Such a metrics collection instrument can be created by your in-house programmers. I don't think that you should/can find anything off the shelf.

0

If I understand your question correctly, you receive datasets which you aren't certain cover 100% of the data. You need some kind of system to be able to confirm that the datasets are complete and otherwise meet your acceptance criteria so that they can be handed off to the next unit, e.g. for analysis.

I'm in the pharma sector, so we have to ensure for our studies that data is accurate for regulatory reasons. This is typically done through quality control checks on the data, followed by quality assurance checks to make sure QC worked as advertised. This can be made more efficient by using electronic data capture systems, but ultimately there has to be some level of human intervention.

Is it possible to use a naming convention for your files so that it would be relatively easy to manually review them to identify obvious gaps?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.