Studies suggest that it typically takes 20 years to go from the discovery of new materials to fabrication of new and next generation devices based on new materials. Many other scientific domains show similar delays. These scientific cycles must be shortened, and it will require a major transformation in how we collect digital data about physical artifacts from scientific instruments, and how we make the digital data available to computational tools for developing new materials, fabricating new devices, and speeding up the discoveries in science. In this talk we present a real-time micro-service operating infrastructure for scientific workflows, named 4CeeD that focuses on the immense potential of capturing, curating, correlating and coordinating digital data in a real-time and trusted manner before fully archiving and publishing them for wide access and sharing.
We will discuss novel cloud-based services, and algorithms that are an integral part of 4CeeD for collecting and correlating data from microscopes and fabrication instruments, modeling of scientific workflows in clouds as micro-services, self-adaptive micro-service-based infrastructure for heterogeneous scientific workflows, and role of edge computing. Specifically, we will show that our micro-service based approach helps to improve flexibility of workflow composition and execution, and enable fine-grained scheduling at task level, considering task sharing across multiple workflows. Furthermore, the self-adaptive micro-service management employs integration of feedback control, deep learning and optimization frameworks to offer resource adaptation without any advanced knowledge of workflow structures. We will also discuss a unique usage of edge computing to enable integration of aging instruments to be integrated into the micro-service operating infrastructure. The evaluation of the 4CeeD system shows robust prediction of resource usage and adaptation under dynamic workloads, real-time service to users to offload, curate and analyze data, and access to diverse instruments.
Joint work with Phuong Ngyuen, Tarek Elgamal, Steve Konstanty, Todd Nicholson
Klara Nahrstedt is the Ralph and Catherine Fisher Professor in the Computer Science Department, and Director of Coordinated Science Laboratory in the College of Engineering at the University of Illinois at Urbana-Champaign. Her research interests are directed toward tele-immersive systems, end-to-end Quality of Service (QoS) and resource management in large scale distributed systems and networks, and real-time security and privacy in cyber-physical systems such as power grid. She is the co-author of multimedia books `Multimedia: Computing, Communications and Applications' published by Prentice Hall, and ‘Multimedia Systems’ published by Springer Verlag. She is the recipient of the IEEE Communication Society Leonard Abraham Award for Research Achievements, University Scholar, Humboldt Award, IEEE Computer Society Technical Achievement Award, ACM SIGMM Technical Achievement Award, and the former chair of the ACM Special Interest Group in Multimedia. She was the general co-chair and TPC co-chair of many international conferences including ACM Multimedia, IEEE Percom, IEEE IOTDI and others. Klara Nahrstedt received her Diploma in Mathematics from Humboldt University, Berlin, Germany in 1985. In 1995 she received her PhD from the University of Pennsylvania in the Department of Computer and Information Science. She is ACM Fellow, IEEE Fellow, and Member of the German National Academy of Sciences (Leopoldina Society).