Case Study: National Data Lake for Higher Education Institutions

Trigyn’s client was a national government agency with the mandate to serve as the primary national-level advisory body with a mandate to ensure the quality of technical education in the country. The client evaluates technical education facilities across the country and supports the development of cohesive educational programs. The client sets standards for the countries 55,000+ higher education institutes, accredits programs and institutions, and ensures the quality of technical education nationwide.

Under this initiative, the client was seeking to consolidate multiple legacy systems to create a unified data collection, storage and analysis platform which would capture all data submitted by higher education institutes across the country on to a single, unified data lake. The new solution was intended to streamline data submission processes, minimize redundant data submissions by higher education institutes, and facilitate easier and more robust data analytics.

To facilitate cost-effective operation, maintenance, and enhancement of the solution, the client mandated the use of open-source technologies to the fullest extent feasible. The solution was required to meet all government IT standards. Trigyn was engaged to design and implement the new solution including custom software development, hardware, training, and ongoing maintenance and support. 

Trigyn Solution:

The scope of the project included the following:

1. Requirements gathering and developing project plan.
2. System design including information architecture, application architecture, infrastructure architecture, UI, and prototype.
3. Comprehensive development, systems integration, and testing of web and mobile versions.
4. Implementation. This phase spanned implementation and operation of new application including:
   - Design, development and testing of the new core application,
   - Supply, installation, configuration, and commissioning of hardware and software for a dedicated infrastructure solution including data centre and separate disaster recovery site,
   - Commissioning of network connectivity,
   - Migration of legacy data,
   - Integration with other government applications,
   - Design, technical and user documentation,
   - Change management and stakeholder training,
   - Operation and maintenance of application and infrastructure as per client service level specifications after delivery.

The solution was built in adherence with client technical, security and privacy standards as well as the following requirements and features:

1. Scalable, secure, robust, highly available, and responsive online system for centralized Data Collection Platform for all higher education institutions.
2. Bilingual UI.
3. UI and workflows for various user types including Students, Faculty, Scholarship/Grant Programs, Committees, Facilities, Infrastructure, Employers, and others. 
4. Workflows to support:
   - students, 
   - faculty, 
   - scholarship and grant programs, 
   - committees, 
   - facilities, 
   - infrastructure, 
   - employers, 
   - feedback system, 
   - participation, 
   - IP filing and grants, 
   - expenses and revenue, 
   - innovation projects and start-ups, and 
   - research publications. 
5. Integration with existing client. systems such as approval, affiliation, accreditation, ranking system, etc. and flexibility to integrate with new systems in the future.
6. All the submitted data is centrally stored in a global data model that captures all possible details related to an institute.
7. Disaster Recovery mechanism to be in a secondary facility.
8. UI for higher education institutes to manage access controls for their downstream users.
9. Capacity and flexibility to adapt to evolving policy and administrative rules.
10. Data Access APIs built on top of the global data model/data lake to allow sharing of relevant data to state-level education authorities, approval bodies, affiliating universities, accreditation bodies, ranking agencies, etc. 
11. Robust data analytics and reporting capabilities to meet the operational and planning needs of the client organization and various government ministries. 
12. Companion mobile application.
13. Seamless integration with legacy platforms via connectors/APIs.

Technologies Used:

Xamarin, Angular, Spring boot, Kubernetes, Kafka, ETL, Druid, NoSQL, Cassandra, Solr, Power BI, Openfire Spark, Oracle SOA Suite, MariaDB, Galera cluster

Outcome:

Trigyn delivered a comprehensive, scalable solution that successfully fulfilled the client’s project goals. 

For more information about Trigyn’s Big Data Analytics Services, Contact Us.