...
During a research project, data takes on different stages of use each with a different storage requirement. Some research projects will use all of these stages, some will only use a few.
Potential Data - data Data that are not yet collected, but there is a plan to store them. This storage could be temporary prior to transfer to a more permanent place, or is immediately transferred from an instrument etc.
Raw Data - This transitional stage includes everything that is collected from the potential stage and could be stored in the same temporary place if deemed not as valuable, or in a more permanent place if keeping raw is determined to be valuableinto a place for processing or pre-processing.
Prepared Data - This stage describes the pre-processing of the raw data that prepares for a model or other processes. This should be stored or transferred to storage that will optimize the process. T
Intermediate Data - This stage is the most temporary of all data, it could be a step in a process that creates these data before processing into final data or simulated data that helps train a model, these data should be stored in a highly performant in read and write operations and backups are not necessarily a requirement.
Final Data - The resulting data of a process. These data tell the story of the research and indicate the results. They should be stored in a safe place if the process is difficult to re-do.
Published Data - These data are the same as the final data, but may be are in a different format optimized for data sharing. Possibly a compressed file stored on a repository.
Archived Data - This stage of data are no longer needed for ongoing research projects but are not deleted. Stored somewhere in “cold” storage in the most cost effective way possible.
...
Assessing the Needs
Two Column Tables are nice ways to separate content/ Background info along with a code example on the same “Slide”. Please notice the table width. This should stop scroll bars from appearing
...
Bullets are nice to include for distinct points
...
yep
...
they
...
sure
...
Stage | Storage Need |
|---|---|
Potential | None yet, but a plan for raw is in place |
Raw | Could be stored in a temporary place or in a more permanent place if keeping raw is determined to be valuable |
Prepared | Should be stored or transferred to storage that will optimize the next process |
Intermediate | Should be stored in a highly performant in read and write operations and backups are not necessarily a requirement |
Final | Should be stored in a safe place if the process is difficult to re-do |
Published | May be in a different format for data sharing, possibly a compressed file stored on a repository |
Archived | Could be stored somewhere in “cold” storage in the most cost effective way possible |
...
Comparing Storage Options
...