Bulk Data Movement with Fedena

Fedena (1)

Data and data analysis plays a vital role in our day today life. Everything we look into is associated with some kind of data or other. Data plays a key role in arriving at any decisions . It support in decision making. In short, data can be defined as information that has been translated into any form that is more convenient to move or process.

Data can be different. It can be either meaningful or meaningless or can be abstract. When it comes to computing, data is the prime component. Every aspects related to computing is associated with some kind of data processing. Conversion of computing problems into executable programs involves analysis, understanding of problem data. Object Oriented Programming makes this conversion much easier by connecting all real world things to objects that have data fields (attributes that describe the object) and associated procedures known as methods.

In Fedena we manage gbs of institution data.  In between fedena’s evolution, we use to get plenty of requests related to data retrieval and data entry. So data management has always been a topic of study.So for the bulk movement of data into and from fedena.Considering the importance of data processing ,what we could do is create an exporting and importing interface. But fedena expands rapidly, its not wise to implement this for every new modules made. So what we needed is an engine to handle data bulk export and import, which can adapt to the changes in fedena easily. Hence we design Data Export and Custom Import plugins which handles bulk import and export of data in a simple manner.

And for the engine design, we exploited the MVC architecture. In MVC architecture the data and logic are separated into different layers. Model represents the physical data in your application. MVC architecture converts the database into model structure with the help of some object oriented principles. Model layer helps in creation and updation of data inside each tables. The data can be captured in the form of attributes and their values of objects. Each tables inside the database will be mapped to one model. We can add more power to the data analysis by creating relationships between data in different tables.

Two engines one handles import and other export. In both we define a particular schema for data for each models. When importing according to the schema the engine creates a csv file which defines headers for the particular model. User downloads it and merges with the data they want to import. The header as I said will be defined by the engine, and the header for a user selected model will be closely associated to the real world object itself. For example a header for Student model will contain first name, last name, admission number, address, batch name etc. While during data export a same schema is set for each models. The user selects the model, accordingly the engine creates export files. The engines makes it easier to extend the functionalities for new module within Fedena.

When it comes to ERP applications data movement has an important role. Almost all ERP applications use extensive set of data for its functioning. The way each of the application process and use data may differ. Fedena custom import and export plugin helps a great deal in data management. Fedena custom import module is primarily developed to move bulk data into fedena. This eases the time consuming data entry process. Fedena custom import plugin supports commonly seen file formats which user can choose as per the requirement. In overall complexity in the data upload is greatly reduced.

Fedena data export module supports the user to export Model data for various purposes. There will be a predefined set of exports (model) available. Model data can be downloaded in either of formats xml or csv. These exported data are of various use. The export operation serves many purposes like backup of certain model, data transfer to some other application, for acquiring meaningful insights from dataset, data driven decision making etc. And overall today’s business, organisational process plannings are based on data. Data is constantly analysed to for evaluation. Different kind of outputs are made out of same set of data. Data exported from Fedena can be used for many of all of those reasons.

This article is written by:

Priyanka TP

Somebody who literally cooks code, Priyanka is kind of girl who works smart in kitchen and on computer with the same zeal. And her codes are equally noteworthy as her delicious recipes.

Share Button