Monitoring, Supervision and evaluation by STDCs

Monitoring, review, supportive supervision and evaluation are important to ensure that the programme is implemented as planned against the stated objectives and desired results. These are intensive technical activities and core function of the STDCs.

The STDCs need to lead activities in this area and support STCs. Designated nodal personnel (Medical officers, Epidemiologist, Statistician, Ni-kshay Operator) should be made accountable for regularly accessing data available from Ni-kshay and other programme information sources. They should also be involved in analyzing the data and preparing analyses and programme performance updates and reports for presentations (PPT) during review meetings. Based on the ongoing monitoring, review and supervisory activities, STDCs should regularly provide action-based feedback to districts and seek timely action taken report from them. 

Details of the processes and approach to be followed in monitoring, supervision and evaluation are available in the NTEP Supervision and Monitoring Strategy (Click here).

The role of STDCs in SM&E are described below.

Programme Monitoring

Programme monitoring means collecting, compiling and analyzing data to determine progress of various activities under the defined thematic areas against the stated objectives at the district and sub-district level on real-time basis and periodically. Monitoring uses a set of core indicators and targets to provide timely and accurate information in order to review performance and inform progress and thus facilitating decision-making processes. This helps us to find timely solutions to the performance gaps- in each district. Programme monitoring is also of paramount importance for ongoing programme planning and implementation. Programme monitoring should be a routine function of STDCs.

The STDC need to access the Ni-kshay data and monitor the completeness, correctness and timeliness of data entry. They have to monitor the district wise performances on a day to day basis using the key indicators suggested by the NTEP. Further periodic Ni-kshay data analysis has to be undertaken for monitoring programme performance.   

Whenever there is a significant under/over achievement or a remarkable variation of a particular outcome indicator, STDCs need to do a 'Root Cause Analysis' by looking at the input, process and output indicators related to it to find out the reason for the same. Using the Ni-kshay data, STDCs also should do the person, place and time analysis to gain insights to identify patterns and anomalies that may indicate reasons for poor or under achievement.

Based on the ongoing review, the STDC needs to

1. Prepare and communicate the feedback with action points to the districts, as and when they find an issue in consultation with STO. The feedback shall include the detailed observations on key indicators, their Root Cause Analysis in short and suggested solutions to overcome the same.
2. Obtain timely action taken reports and in-turn monitor the change in performance after the action.
3. Purposively select Districts/ TB units (TUs)/ health facilities to be visited during supportive supervision, and evaluation.
4. Identify priority performance themes and geographies (Districts/ TUs) to be reviewed during periodic review meetings.

Programme Reviews

Review meetings are conducted periodically aiming at identifying areas and actions for improving the programme performance. They are essential for quality improvement and sharing of best practices. STDCs need to support STC in the planning and conduct of monthly review meetings of the districts state in the following ways.

Planning and preparation for reviews is a crucial step that STDCs need to undertake in consultation with STCs. Thorough preparation and advance planning of agenda items (Programmatic themes and operational areas) needs to be done prior to the meeting based on inputs obtained from ongoing monitoring.  

During the review meetings, STDCs should guide the discussions. such that they are based on objective criteria, and discussions progress in a consultative mode towards solutions. STDCs should  ensure that the discussions are not limited only to outcomes, but also to inputs, processes and outputs. STDCs also should ensure that districts share their experiences, challenges, and best practices to facilitate cross-learning platform.

STDCs need to ensure that the review concludes with action-oriented feedback directed at specific responsible persons. All the feedback should be communicated to the responsible authorities within one week of the review meeting. All the emerging feedback from the review meeting should be consolidated as minutes meeting. STDCs also need to track the actions taken by all concerned based on the decisions made in the meeting/ feedback provided. 

Supportive Supervision

Supervision is a systematic process for increasing efficiency of the health personnel by updating their knowledge, perfecting their skills, improving their attitudes towards their work and increasing their motivation. It is thus an extension of training.

Supervisory visits by STDCs should be a collaborative effort to identify problems and find solutions. The effort by the supervisory team, should be issue based and directed at provide ongoing support for solving problems and to overcome difficulties. Feedback provided during the visit should be constructive (with practical solutions) and not mere observations on performance. It should aim to build capacity of the health staff to implement the programme procedures correctly with high level of efficiency, to assess HR and training needs and to ensure data and service quality. At the end of the visit, the staff should be motivated and encouraged to effect solutions and bring improvements in programme performance.

Selection of districts: STDCs should conduct district wise supportive supervisory visits oriented towards the performance of the district as a whole, and in-turn evaluate performance and build the capacity/ competencies of the DTO and other NTEP staff and health functionaries. Selection of the district for supervision should be made based on the routine monitoring done by STDCs. Districts which require support need to be given preferences and shall be visited more frequently. However, selection should be done such that visits should cover all districts in the state at least once in a year.

Supervisory Team: A minimum of two members consisting of at least one medical officer from STDC/ RTPMC should visit the district for supportive supervision. And may be accompanied by STDC Director/ head of institution/ Senior Supervisory Staff from the state.

I. Conducting the visit in the district:

Key thrust programme operation areas for a supervisory visit should be purposively decided prior to the visit based on ongoing monitoring of the district performance. The institutions and corresponding staff to be visited may be decided based on the same. For example, if the reason for selection of a district is low treatment success rate from private sector, then high load private sector health facilities, current beneficiaries, their linked TU staff, and district PPM co-ordinator would be visited.

During the supervisory visit, data quality assessment in the selected focus areas should be done to assess correctness, completeness and timeliness. This would include verifying/ comparing data recorded in Ni-kshay (dates, categories, quantities) against the true/ actual value identified based on the context (recall/ records from beneficiary, hospital records etc.)   

II. Visit Planning:

The Medical Officer (SM&E) of the STDC/ RTPMC should prepare in advance the monthly supervisory visit plan along with the focus areas for each district and include the same in Advanced Tour Program (ATP). A minimum of two districts should be visited by the STDC supervisory staff every month. For better efficiency, supportive supervisory visits could be clubbed with EQA visits by IRL. The monthly visit plan (ATP) should be submitted to the STDC Director before 25th day of the previous month. The STDC Director in-turn should communicate it to STO, the corresponding districts and the linked National Institute before the first official day of the month.

The supervisory visit should be made for at least for 3 days or more working days in each district. Based on the selected thrust area, institutions, staff and beneficiaries to be visited, the Medical Officer (SM&E) of the STDC may consult district to chalk out a suitable travel plan in advance. Based on the plan the Medical Officer (SM&E) needs to arrange suitable travel logistics with the approval of the STDC Director.

III. Output of a Supervisory visit:

At the end of the supervisory visit, feedback should be prepared and it should be specific action oriented along with supporting observations, with timelines. The observations and recommendations of supervisory visit should be discussed with the DTO and team and also for any administrative issues the districts/ State Health authorities may also be appraised. It should be submitted to the relevant authority(District TB officer, CMO, STO etc) who should take the action within 7 days of the end of the visit. STDCs should ensure that the each recommendation is an activity/task for the district/state to perform. STDCs should advocate to solve any issues requiring actions at the state level. STDCs should also monitor the actions taken against each feedback and close the feedback once the task is satisfactorily accomplished.  

Internal Evaluation

Internal Evaluation forms an integral component of NTEP supervision and monitoring strategy. It acts as a tool to evaluate if good programme practices are adopted and quality services are provided to the community. These evaluations also offer an opportunity to look into all aspects of programme critically and swiftly. These activities help programme managers in understanding determinants of good as well as poor performance for replication of good practices in other districts and take appropriate measures for their improvement.  

The STDCs should plan and lead the State Internal Evaluations. STDCs need to prepare a plan so as to cover all the districts in the state at least once in 3 years. Members of the team make plans and formats for evaluation should be based on the NTEP SM&E guidelines.