My suggestions on coping with big data challenges
– Missing data: It depends on what type of data you want to use from EMR. If you want to do prospective research that sources of data come from EMR. You can solve this problem by setting the desired data to be a mandatory field for a physician to fill it in EMR. However, if you want to do retrospective research from EMR data, the missing data should be a big problem like other retrospective research. You have to downsize the sample size, use only an EMR with complete data of interests or use statistical analysis technic to cope with this problem.
-Selection bias: Doing a research with data from EMR is the observational study that has a bias more than RCT. the research methodology for the research on big data should be matched with the nature of data sources to maximized the value of observational data and minimized bias.
-Data analysis and training: There is a new frontier for working on big data and come out with useful information that can make a better health care system. We have to train a lot of data analysts of big data because this type of data is unlike other data that we familiar with. and we also may need machine learning to generate useful information.
-applicability of the results: For the implementation of results, traditionally the results of the research will apply to the same population as stated in research population. While research on the big data, like EMR, we are difficult to define what is the character of the research population. In my opinion, we can cope with this problem by specifically identified what is the inclusion criteria. The well-defined inclusion criteria of the research population will help people to confidently apply research results into clinical practice.
-Privacy and ethical issue always a concern. the rule and regulation should be applying for data security and privacy.