- Open Access
- Total Downloads : 318
- Authors : Brenno Barbosa De Alencar Fernandes , Jadson Vieira De Oliveira , Daniel Candido De Oliveira
- Paper ID : IJERTV6IS100121
- Volume & Issue : Volume 06, Issue 10 (October 2017)
- DOI : http://dx.doi.org/10.17577/IJERTV6IS100121
- Published (First Online): 23-10-2017
- ISSN (Online) : 2278-0181
- Publisher Name : IJERT
- License: This work is licensed under a Creative Commons Attribution 4.0 International License
Evaluation of the Implementation of the Electronic Information System from the Perspective of Users: a Case Study at the Federal Institute of Education, Science and Technology of Tocantins
Brenno Barbosa De Alencar Fernandes
Degree in Internet Systems Federal Institute of Education,
Science and Technology of Tocantins Palmas TO, Brazil
Jadson Vieira De Oliveira Federal Institute of Education, Science and Technology of Tocantins Palmas TO, Brazil
Daniel CÃ¢ndido De Oliveira Federal Institute of Education, Science and Technology of Tocantins Palmas TO, Brazil
AbstractThis research aims to investigate the implementation process of the Sistema EletrÃ´nico de InformaÃ§Ãµes (Electronic Information System) – SEI, at the Instituto Federal de EducaÃ§Ã£o, CiÃªncia e Tecnologia (Federal Institute of Education, Science and Technology of Tocantins) – IFTO, focusing on the IFTO public servants. For this, a search was made in the specialized literature, in which success factors related to user satisfaction in the system implementation were extracted. Then, to confirm the state of the art, a descriptive case study was developed. The data were collected through a judgment matrix and later treated in order to validate the presented proposal.
Information technology has contributed to the improvement of the relationship between government, citizens and representative groups of society  . In the last years the public power has been adopting these resources as a form of improvement in the provision of services to the citizens.
One of the systems that has been widely adopted in organs of different spheres is the Electronic Information System (SEI), which aims to implement the electronic process in place of existing physical processes, seeking greater agility, speed, transparency and efficiency in services provided . The SEI was adopted within the scope of the IFTO in order to comply with the Presidential Decree No. 8,539, of October 8, 2015, which provides for the use of electronic means to carry out the administrative process .
The SEI was developed by the Federal Regional Court of the 4th Region (TRF4), and disseminated to other federation bodies, free from charge, and an official letter was sent to the Ministry of Planning, Development and Management (MPDG), formally showing intention to obtain the right to use the system. Subsequently, the Agreement of Technical Cooperation is signed, which formalizes the assignment of the use of the system, and alongside with this term, the Plan of Work must also be signed, where the initial planning for the implementation of the system in each body or entity is defined .
In the process of implementing an Information System (SI) it is not possible to control all present factors in the implementation, critical factors are crucial for the success or failure in an information system implementation      . Users satisfaction is shown as a matter of prime importance, once the perception over the benefits provided by the system might improve its performance and productivity       .
This work aims to be a useful tool in the aid of decision making and IT governance. For such purpose, it has IFTO as focus and it presents, in this context, an evaluation of the impacts in implementing SEI in IFTOs scope under the users perspective. In specific, this work aims to present a theoretical survey that substantiates the research, as well as presenting users opinion, from which the collected data referring to the system implementation and using experience were verified through a judgment matrix and a proper statistic treatment. Therefore, at the end of this work it shall be possible to compare the state of the art and the state of the practice, relating critical factors yet to be developed in order to assure its continue usage with efficiency.
Performing a descriptive research aims to expose the characteristics of a certain population or phenomenon, and in order to reach this goal it is necessary that the investigator acquires several data about the object of study  , allowing reality to be analyzed from the participants perspective providing a global , coherent and thorough view to the investigator  . This study focuses on analyzing a certain situation, which must have relevance and be representative, picturing reality without the researchers interference.
By performing the theoretical survey the key words users satisfaction and information systems were used, and from the works found, the ones having a theme related to the project were selected. From these, it was possible to extract methods that contained efficiency proof in the analysis to be produced.
Based on the work of several researchers,  defined four dimensions: (a) task productivity, (b) innovation, (c) customer satisfaction and finally (d) managerial control, which combined, help describing the impact of an application on the individuals of an organization . From this assumption, this research focuses on the dimension of users satisfaction.
In order to allow the evaluation of the impact of the SEI implementation, a questionnaire was applied to the public servants with closed questions, and the answers were organized according to the Likert scale, which includes statements on a scale between "totally agree" and "totally disagree"  . In addition, according to , we used statistical methods to perform a more sophisticated analysis of the data.
The questions had as goal to verify the effectiveness of the implementation of the SEI in IFTO from the users perspective, involving several questionings observed in the theoretical cut- offs, which were grouped in four critical factors that have influence on the users satisfaction, being able to be observed in Table I.
TABLE I. CRITICAL FACTORS ON THE PROCESS OF IMPLEMENTING AN INFORMATION SYSTEM BASED ON USERS SATISFACTION
    
Reliability of information
    
Training and education
Monitoring, control and support
  
Fig. 2. Age range.
Fig. 1 presents the conceptual model for validation of our hypotheses to be verified throughout the work. Critical factors have a greater or lesser effect on users satisfaction.
Fig. 1. Conceptual model.
TABLE II. QUESTIONNAIRE RELIABILITY ACCORDING TO CRONBACH ALPHA
Bigger than 0,9
Between 0,8 and 0,9
Between 0,7 and 0,8
Between 0,6 and 0,7
Between 0,5 and 0,6
Smaller than 0,5
a. Source 
The research had the participation of 347 public servants categorized by age range, academic formation and assignment time in the institution as shown in fig. 2, 3 and 4.
The data were collected through an electronic questionnaire, madeavailable to the institution's public servants. IFTO has a functional population of 1221 public servants, of which 354 participated. In order for the answers to be consistent and relevant to this work, some control questions were inserted. From these, 347 were validated for statistical treatment. The achieved proportion of validated respondents, relative to the total population, provides the survey a 95% confidence level, with a sample error margin of 4.46% .
In order to determine a level of confidence of each critical factor and the correlation of the items we used the Cronbach's alpha value . Table II presents the classification levels of the Cronbach Alpha.
Fig. 3. Academic formation.
Fig. 4. Assignment time.
The statistical treatment was applied to reduce the subjectivity of the answers, so that the results of the research would be able to reflect the reality of the whole. We considered satisfied users those who had an average of each critical factor between 5 and 7, based on the Likert scale. In order to evaluate the influence of the critical factors, according to table 1, in the users satisfaction, follow the analysis below.
Looking at Table III we see that the user's perception regarding the critical factor business processes presents a weighted average of 5.15. Since the item cost reduction by archiving and storage presents higher average of 5.78 and lower standard deviation. This critical factor reached 66.92% satisfaction of the participating users.
TABLE III. CRITICAL FACTOR BUSINESS PROCESSES
Average of items
Average of the critical
P1 – Was there a decrease in
P2 – Was the time for carrying out the activities
reduced by using the system?
P3 – Did IFTO reduce the costs of filing and storing
P4 – Did SEI make your
service more productive?
P6 – Is SEI useful in relation
to your work needs?
Table IV presents the critical factor "Reliability of information", which obtained approximately 77% approval from its users. The item P5 obtained an average of 5.99, being the highest among all the items surveyed.
TABLE IV. CRITICAL FACTOR RELIABILITY OF INFORMATION
Average of items
Average of the critical
P7 – Whenever necessary do you have access to the
information in SEI?
P5 – Is the information
provided by SEI reliable?
P8 – Do you consider the
P9 – Does SEI respond
quickly to your requests?
A control question was used to separate the answers from those who participated in the trainings offered to use the system, totaling 57.7% of the interviewees. It can be seen from Table V that P10 received the average 4.24, this was the lowest average of all the research, and it may be related to the short time available for training.
TABLE V. CRITICAL FACTOR TRAINING AND EDUCATION
Average of items
Average of the critical
P10 – Was training for SEI
P11 Does the training environment (http://treinamento.sei.ifto.ed u.br) aid in learning to use the
P12 – Was the material
available for study satisfactory?
P13 Has the guidance page on the use of SEI, (http://sei. ifto.edu.br/site-sei/), assisted
in the use of the system?
Based on a control question, as well as with the critical factor "Training and education", the data of those who used support were analyzed in some way, being that they number approximately 65% of the respondents. Based on these data, Table VI was created, analyzing the high degree of satisfaction with the offered support, with the average being 5.68.
TABLE VI. CRITICAL FACTOR MONITORING, CONTROL AND SUPPORT
Average of items
Average of the critical
P14 – Have your demands
always been attended?
P15 – Were the demands
About 35% of the participants did not need support in any way, of these 57% feel satisfied with the system.
A. Critical factors level of importance
Through the results it is possible to demonstrate, through Table VII, the relative importance of the critical factors. Given that these data can be used to assist in decision making and IT governance, in order to guarantee efficient continued use of the system.
TABLE VII. CRITICAL FACTOR MONITORING, CONTROL AND SUPPORT
Monitoring, control and
Reliability of information
Training and education
This paper aims to evaluate the influence of critical factors on the users satisfaction dimension in helping decision making and IT governance. For this, a survey of the systems evaluation practices regarding the users satisfaction was carried out in the light of the specialized literature. The study sought to ascertain if the state of the art reflected in the state of the practice through an electronic questionnaire. For that, an exploratory research was carried out with direct users of the SEI – IFTO public servants – that would be able to respond on the effects, based on their experiences and technical knowledge.
The data collected through the electronic questionnaire validated the presented proposal. Critical factors were evaluated from the Likert scale and impact weights were assigned to them. From this, a positive response was obtained regarding the hypothesis tested by this study, and it was verified that the practices focused on users satisfaction cause a greater or lesser effect on the prospecting of technology in the value chain of systems implantation, where practices were considered to have a moderate effect, tending to be high, on the dependent variables, and even some of them presenting a minor impact, no practice had a below-moderae impact.
It was possible to perceive a high degree of satisfaction of the users with the system in question. But in order to be able to attest this affirmation, new researches must be carried out periodically, later through a comparative between the surveys we can affirm whether the implantation was successful. However, it is important to emphasize that the research was limited to the implementation of SEI in IFTO, which makes it possible and recommended to expand the study to other experiences, as well as extend to other specialists with technical and scientific knowledge on the subject addressed here.
Akkermans, H. and Helden, K. (2002). Vicious and virtuous cycles in erp implementations: a case of interrelations between critical success factors. European Journal of Information Systems.
Albernaz, C. M. and Freitas, A. L. P. (2010). Um modelo para avaliaÃ§Ã£o da qualidade de serviÃ§os de suporte de tecnologia da informaÃ§Ã£o. Anais do XXX ENEGEP.
Albertin, L. A. (2001). Valor estratÃ©gico dos projetos de tecnologia da informaÃ§Ã£o. Revista de AdministraÃ§Ã£o de Empresas, 41(3):4250.
Anjos, G. M. A. A. and Ezequiel, V. d. C. (2011). Cidadania Virtual: O EspetÃ¡culo do Governo EletrÃ´nico. Estud. sociol, 16:5976.
Balbe, R. d. S. (2010). Uso de tecnologias de informaÃ§Ã£o e comunicaÃ§Ã£o na gestÃ£o pÃºblica: exemplos no governo federall. Revista do ServiÃ§o PÃºblico, 61:20.
Bergamaschi, S. (2001). Fatores crÃticos de sucesso para implementaÃ§Ã£o de sistemas de gestÃ£o empresarial. Atlas, SÃ£o Paulo.
Brasil (2015). Decreto N o 8.539, de 8 de outubro de 2015. Dispo- nÃvel em http://www.planalto.gov.br/ccivil_03/Ato2015- 2018/2015/Decreto/D8539.htm. Accessed in: 2017-07-10.
Catersels, R., Helms, R., and Batenburg, R. (2010). Exploring the gap
between the practical and theoretical world of erp implementations: results of global survey. Proceedings of the International Conference on Research and Practical Issues of Enterprise Information Systems (CONFENIS).
Esteves, J., Casanovas, J., and Pastor, J. (2003). Using the partial least squares (pls) method to establish critical success factors interrelations in erp imple- mentation projects. Proceedings of the Americas Conference on Information Systems (AMCIS).
Ferreira, A. P. D. and Ferreira, L. (2008). Medindo a SatisfaÃ§Ã£o dos usuÃ¡rios de um Sistema Integrado de GestÃ£o. SEGeT SimpÃ³sio de ExcelÃªncia em GestÃ£o e Tecnologia, pages 1 11.
Fitzsimmons, J. A. and Fitzsimmons, M. J. (2000). AdministraÃ§Ã£o de ServiÃ§os: operaÃ§Ãµes, estratÃ©gia e tecnologia de informaÃ§Ã£o. Bookman, Porto Alegre, 2 edition.
Fonseca, J. J. S. (2002). Metodologia da pesquisa cientÃfica. UEC, Fortaleza.
George, D. and Mallery, P. (2002). SPSS for Windows step by step: A simple guide and reference, 11.0 update. 4th Boston: Allyn & Bacon.
Gil, A. C. (1999). MÃ©todos e tÃ©cnicas de pesquisa social. Atlas, SÃ£o Paulo, 5 edition.
Governo-Federal (2017). Processo EletrÃ´nico Nacional – Sistema EletrÃ´nico de InformaÃ§Ã£o – PEN – SEI. DisponÃvel em https://softwarepublico.gov.br/social/sei. Accessed in: 2017-07-10.
Hora, H. R. M., Monteiro, G. T. R., and Arica, J. (2010). Confiabilidade em questionÃ¡rios para qualidade: Um estudo com o coeficiente alfa de cronbach. Produto & ProduÃ§Ã£o, 1(2):85103.
Klein, M. M. (2012). Processo eletrÃ´nico na JustiÃ§a Federal da quarta regiÃ£o: verificaÃ§Ã£o do resultado de sua implementaÃ§Ã£o.
Laudon, K. C. and Laudon, J. P. (2007). Sistemas de informaÃ§Ã£o gerenciais. Prentice Hall, SÃ£o Paulo, 7 edition.
Oliveira, M. M. d., Ponchio, M. C., Neto, M. S., and Pizzinatto, N. K. (2009). AnÃ¡lise dos fatores de resistÃªncia na implantaÃ§Ã£o de sistemas de informaÃ§Ã£o na manufatura de eletrÃ´nicos. JISTEM Journal of Information Systems and Technology Management, 6:507524.
Roesch, S. M. A. (2012). Projetos de estÃ¡gio e de pesquisa em administraÃ§Ã£o: guia para estÃ¡gios, trabalhos de conclusÃ£o, dissertaÃ§Ãµes e estudos de caso. Atlas, Atlas, 3 edition.
Salvalaggio, A. P. d. S. (2015). AvaliaÃ§Ã£o do Sistema Erp AtravÃ©s da SatisfaÃ§Ã£o e Produtividade dos UsuÃ¡rios Finais da Empresa Vitasons Centro de Apoio Auditivo. pages 1 79.
Santos, G. E. d. O. (2017). CÃ¡lculo amostral: calculadora on-line. DisponÃvel em http://www.calculoamostral.vai.la. Accessed in: 2017-07-10.
Severino, A. J. (2007). Metodologia do trabalho cientÃfico. Cortez, SÃ£o Paulo, 23 edition.
Somers, T. M. and Nelson, K. (2001). The impact of critical suc- cess factors across the stages of enterprise resource planning implementations. Proceedings of the Hawaii International Conference on System Sciences.
Sun, Y. A., Yadzani, A., and Overend, D. J. (2005). Achievemente asses- sment for enterprise resource planning (erp) system implementations based on critical success factors (csfs). International Journal of Production Economics, 98:189203.
Torkzadeh, G. and Doll, W. (1999). The development of a tool for measuring the perceived impact of information technology on work. Omega, 27(3):327339.
TriviÃ±os, A. N. S. (1987). IntroduÃ§Ã£o Ã pesquisa em ciÃªncias sociais: a pesquisa qualitativa em educaÃ§Ã£o. Atlas, SÃ£o Paulo.
Umble, E., Haft, R., and Umble, M. (2003). Enterprise resource planning: Implementation procedures and critical success factors. European Journal of Operational Research, 146(2):241257.
Valente, N. T. Z. and Riccio, E. L. (2004). ImplementaÃ§Ã£o de ERP em Pequenas e MÃ©dias Empresas: Estudo de Caso em Empresa do Setor da Cons- truÃ§Ã£o Civil.
Xue, Y., Liang, H., Boulton, W. R., and Snyder, C. A. (2005). ERP imple- mentation failures in China: Case studies with implications for ERP vendors. International Journal of Production Economics.