Fall 2017 - Science Gateway Architectures

  • Course: CSCI-B 649, Topics in Systems, Computer Science, School of Informatics and Computing, Indiana University
  • Instructors: Marlon Pierce, marpierc@iu.edu; Suresh Marru, smarru@iu.edu
  • Associate Instructors: Sneha Tilak; Eldho Mathulla

  • Class Schedule Tuesdays and Thursdays from 4 pm to 5.15pm in I2 (Informatics East) Room 150
  • Office Hours On Request
  • HipChat Instant Messaging https://www.hipchat.com/g6Use7j8w

Course Structure

  • Course Goal: students will, working individually, learn and apply modern distributed computing concepts to Apache Airavata and contribute them to the code base.
  • Assignments due every two weeks.
  • All assignments are individual assignments.
  • The course will be split into two parts
    • Part 1: Learning basic distributed computing concepts, microservices, DevOps etc
    • Part 2: Applying what you have learned to Apache Airavata


The course will be taught by Marlon Pierce and Suresh Marru, who lead the Pervasive Technology Institute’s Science Gateways Research Center and are project management committee members for the Apache Airavata open source software.


  • Homework Assignments:
    • Each assignment is worth 10 points
      • You will get 8 points for your submission
      • You will get 1 point for peer reviewing other submissions
      • 2 peer review assignments
    • There will be 8 assignments
    • Assignments will be graded on functionality of the submission, which must be documented in your wiki
    • Each assignment is submitted as a Wiki entry in GitHub
    • Each student gets a github repo in https://github.com/airavata-courses
    • Each assignment must be submitted for peer grading for 1 week, then submitted for final grading after two weeks
  • Presentations and Reports
    • Each student makes a presentation (5 points)
    • Each student submits a report (5 points)
    • Midterm: 10 points
    • Final: 10 points

Apache Airavata Project Themes

  1. Load balancing and fault tolerance of the API Server
  2. Expanding Airavata microservice architecture with new services
    1. CI/CD for new services
    2. Allocation management
    3. Resource status, load, etc
    4. Postprocessing pipelines
  3. Containerizing and orchestrating Airavata deployments
  4. Airavata CI/CD
    1. Resource provisioning
    2. CI/CD
    3. Blue-green deployments
    4. Testing and assurance
  5. Workflows, task orchestration, scheduling, parameter sweeps
  6. Improving interactions with remote resources:
    1. Reducing errors with submissions,
    2. Improving monitoring,
    3. Application-specific monitoring
    4. Running batch cloud applications
  7. Logging, searching, and event detection
    1. Comprehensive, consolidated logging for all Airavata production services
    2. Logs for tenant admins
    3. Search, analytics, event detection
    4. Log visualization
  8. User interfaces and user environments
    1. Jupyter, Django, and related stuff
    2. Application toolkits
    3. Visualization, science desktop integration
  9. Data mining as a back end application
  10. Data management and file transfer


During the course, instructors will provide references to journal and conference papers. A good understanding of concepts discussed in these referred papers will greatly help in absorbing the course material.

Beginner Materials

Open Collaboration

  • Reuse and building upon ideas or code are major parts of modern software development. As a professional programmer you will never write anything from scratch. This class is structured such that all solutions are public. You are encouraged to learn from the work of your peers. We won’t hunt down people who are simply copying-and-pasting solutions, because without challenging themselves, they are simply wasting their time and money taking this class.

  • Please respect the terms of use and/or license of any code you find, and if you reimplement or duplicate a design or code from elsewhere, credit the original source.