Testing data on perceptions obtained from Likert Scales discrete visual analog scales for reliability and validity – the use of Cronbach’s alpha 4.24 5 4.00 5 Note: Mode scores are sh
Trang 1Assessment of capacity improvement within IPSARD/CAP: Baseline survey of knowledge, skills and attitudes (KSA)
compared to end-of-project survey of KSA
1 Introduction and method
A baseline survey of IPSARD/CAP staff was conducted in August 2007 at the commencement of the project The survey was designed to explore the baseline knowledge, skills and attitudes (KSA) of the IPSARD/CAP researchers in areas related to the research work and training that would be conducted during the project
An end-of-project assessment using the same survey was conducted in April 2010 The end-of-project survey is shown in Appendix I The baseline survey was identical except that it did not include the introductory Questions 1, 2 and 3, or Questions B14 and B15 Table 1 shows the distribution of respondents within IPSARD/CAP who answered the baseline and end-of-project surveys
Table 1 Distribution of respondents within IPSARD/CAP completing the baseline and end-of –project assessments for CARD Project 030/06 VIE
Center for Agricultural Policy 10 (10)a
a In April 2010, the 10 CAP respondents were already counted as members of Market Analysis Unit and Rural Development Unit
Respondents (16) who completed the end-of-project survey had worked on average for IPSARD/CAP for 4.7 years (the median was 3 years), and it is probable that some
of those who completed the survey had not completed the baseline survey There had been staff turnover at IPSARD/CAP over the duration of the project, and staff numbers were less at the end of the project than at the beginning Several staff members who had worked on the CARD project during 2007/08 had left the organisation during 2009/10 (two of these however completed the end-of-project survey)
Of the 16 respondents, three said they had been “very involved” in the project, two had been “relatively involved”, three had been “somewhat involved”, three had been
“a little involved” and five said they had “not been involved” Eleven respondents said they had attended a CARD project training, and three others said they were
“unsure” if they had attended a CARD training This is understandable in the context
of the long project duration and the many and varied training sessions in which IPSARD/CAP staff participate
The surveys aimed to assess capacity in knowledge, skills and attitudes in areas directly related to the research work undertaken for the CARD project Respondents were asked to rate:
Trang 2• their level of knowledge in 10 subject areas;
• their level of confidence to perform 15 specified tasks; and
• their attitude to 18 different statements about policy perspectives relevant to the livestock sector
The results are given in Sections 2, 3 and 4 of this report
1.2 Extraneous factors affecting the capacity assessment
Although the sample is not exactly the same for the baseline and end-of-project surveys, this can still be regarded as a valid assessment of changes in knowledge, skills and attitudes (for subjects directly related to the CARD project) within IPSARD/CAP over the duration of the CARD project
Staff at IPSARD/CAP had been involved in many other projects during this period, and these projects are likely to have had an impact on capacity In particular, staff had been involved in three other projects involved with the livestock sector These were:
• ACIAR Project LPS/2005/063 “Improving the Competitiveness of Pig Producers
in an Adjusting Vietnam Market” from 2006 and on-going
• HPAI insurance project with the Japanese Research Institute of Economy, Trade and Industry in 2008 In this project CAP staff supervised a survey of 2,100 poultry and other livestock producers in four provinces but did none of the analysis
• Socio-economic impacts of HPAI outbreaks and control measures on small-scale and backyard poultry producers in Asia – sponsored by the Canadian International Development Research Centre (IDRC) between 2007 and 2009
All these projects involved survey work in the livestock sector However, none of the other projects were specifically involved with the livestock feed production sector
2 Changes in knowledge
In Section A of the survey, respondents were asked to rate their knowledge of 10 different subject areas The average rating and mode scores for the baseline and end-of-project surveys are shown in Tables 2 and 3 The rating scale used was: 1 = very good, 2 = good, 3 = so-so, 4 = poor, 5 = none
2.1 Overall comparisons of baseline and end-of-project results
Overall comparisons are shown in Table 2 Nine out of the ten assessed knowledge areas had improved (based on the average rating score) or remained the same in the end of project survey compared to the baseline The exception was knowledge of the
“indicators of competitiveness of firms” (baseline 2.76, project end 2.81) This is not a good result for the project as competitiveness was a focus of the research work, but perhaps reflects the complexity of assessing competitiveness of SMEs in the livestock feed sector However, the remaining results indicate the likely positive input of the project work in a number of knowledge areas that were key research areas for the project
Trang 3Table 2 Comparisons of average and mode baseline and end-or-project rating scores
for ten knowledge areas (Scale: 1 = very good, 2 = good, 3 = so-so, 4 = poor, 5 =
none.)
Avg/Mode Rating Score
1 Principles of the economics of supply/value chains 3.21 (3) 2.50 (3)
2 The role of SMEs in the livestock feed sector in Vietnam 2.83 (3) 2.38 (2)
3 Indicators of the competitiveness of firms 2.76 (3) 2.81 (3)
4 The theory of transactions costs 3.38 (3) 3.38 (4)
5 Government policies affecting the livestock feed sector in
6 Principles of industrial organization in a market economy –
e.g market power, monopolies, economies of scale/scope,
vertical integration 3.17 (3) 2.81 (3)
7 Implications for the livestock sector of Vietnam’s acceptance
into the WTO 3.28 (3) 2.69 (3)
8 Principles for designing survey questions to minimize bias 2.90 (3) 2.13 (2)
9 Survey sampling techniques 2.97 (3) 2.25 (3)
10 Testing data on perceptions obtained from Likert Scales
(discrete visual analog scales) for reliability and validity – the
use of Cronbach’s alpha 4.24 (5) 4.00 (5)
Note: Mode scores are shown in (in parentheses)
a Survey sample for the Baseline measure was 29
b Survey sample for the End-of-project measure was 16
Five knowledge areas improved on average overall by a score of more than 0.5,
including:
• Principles of the economics of supply/value chains;
• Government policies affecting the livestock feed sector in Vietnam;
• Implications for the livestock sector of Vietnam’s acceptance into the WTO;
• Principles for designing survey questions to minimize bias; and
• Survey sampling techniques
A further two knowledge areas improved on average overall by a score of more than
0.3, including:
• The role of SMEs in the livestock feed sector in Vietnam; and
• Principles of industrial organization in a market economy – e.g market power,
monopolies, economies of scale/scope, vertical integration
Mode scores (i.e the most commonly selected rating) were mostly the same for the
baseline and end-of-project surveys The mode score improved from “so-so” to
“good” for two subject areas: knowledge of “the role of SMEs in the livestock feed
sector in Vietnam” and knowledge of “principles of designing survey questions to
minimize bias” The mode score for knowledge of “theory of transactions costs”
decreased from “so-so” to “poor”
Trang 42.2 Comparisons based on respondents’ level of involvement in
the project
A further comparison was made between respondents with different levels of
involvement in the project Scores for those respondents who nominated themselves
as either “very involved”, “relatively involved” or “somewhat involved” were
compared with scores from respondents who said they were either “a little involved”
or “not involved” The sample of 16 was evenly split between these two categories
The results are shown in Table 3
Table 3 Comparison of average and mode rating scores for 10 knowledge areas:
baseline results and end-of-project results for those respondents involved and
non-involved in the project (Scale: 1 = very good, 2 = good, 3 = so-so, 4 = poor, 5 =
none.)
Avg/Mode Rating Score
1 Principles of the economics of supply/value chains 3.21 (3) 2.50 (3) 2.50 (2,3)
2 The role of SMEs in the livestock feed sector in Vietnam 2.83 (3) 2.00 (2) 2.75 (2)
3 Indicators of the competitiveness of firms 2.76 (3) 2.38 (3) 3.25 (3)
4 The theory of transactions costs 3.38 (3) 3.50 (3,4) 3.25 (4)
5 Government policies affecting the livestock feed sector in
Vietnam 3.31 (3) 2.38 (3) 2.88 (3)
6 Principles of industrial organization in a market economy
– e.g market power, monopolies, economies of
scale/scope, vertical integration 3.17 (3) 2.75 (3) 2.88 (3)
7 Implications for the livestock sector of Vietnam’s
acceptance into the WTO 3.28 (3) 2.63 (3) 2.75 (3)
8 Principles for designing survey questions to minimize
bias 2.90 (3) 1.63 (2) 2.63 (3)
9 Survey sampling techniques 2.97 (3) 2.13 (2,3) 2.38 (3)
10 Testing data on perceptions obtained from Likert Scales
(discrete visual analog scales) for reliability and validity –
the use of Cronbach’s alpha 4.24 (5) 4.00 (5) 4.00 (5)
Note: Mode scores are shown in (in parentheses)
a Survey sample for the Baseline measure was 29
b Survey sample for the End-of-project Involved (End-I)measure was 8
c Survey sample for the End-of-project Not Involved (End-NI) measure was 8
The average ratings were lower (i.e better knowledge) for the involved group
compared to the non-involved group in seven of the ten knowledge areas The score
difference was lower by 0.5 or more in four areas:
• The role of SMEs in the livestock feed sector in Vietnam
• Indicators of the competitiveness of firms
• Government policies affecting the livestock feed sector in Vietnam
• Principles for designing survey questions to minimize bias
These are all knowledge areas very relevant to the CARD project
The mode scores for those involved and not-involved in the project were also different
in a number of cases These were:
Trang 5• Principles of the economics of supply/ value chains The non-involved group had
a bimodal score of “good/so-so” compared to the mode of “so-so” for the involved group
• The theory of transactions costs The involved group had a bimodal score of
“poor/so-so” compared to the mode of “poor” for the non-involved group
• Principles of designing survey questions to minimize bias The involved group had
a mode of “good” compared to “so-so” for the non-involved group
• Survey sampling techniques The involved group has a bimodal score of
“good/so-so” compared to the mode of “so-“good/so-so” for the non-involved group
3 Changes in skills
In Section B of the survey, respondents were asked to rate their confidence to perform
15 specified tasks The average rating and mode scores for the baseline and end-of-project surveys are shown in Tables 4 and 5 The rating scale used was: 1 = very confident, 2 = confident, 3 = somewhat confident, 4 = a little confident, 5 = not confident
3.1 Overall comparisons of baseline and end-of-project results
Overall comparisons are shown in Table 4 All the assessed skill areas were rated as improved in the end-of-project assessment compared to the baseline Two areas
(“Design a database in Access” and “Use Stata for data analysis”) were not included
in the baseline, but were assessed in the end-of-project survey as work/training had been done in these skill areas The results indicate the likely positive input of the project in a number of skills which were key components of the project
Four skill areas improved on average overall by a score of more than 0.7, including:
• Investigate the supply chain for a product;
• Design a survey to map/describe a supply chain;
• Produce descriptive statistics from survey data; and
• Estimate the competitiveness of a firm operating in the livestock feed sector
This last skill area (“estimate the competitiveness of a firm”) which is rated by
respondents as being considerably improved is contrary to the result obtained in Section A, where respondents did not rate their knowledge in this area as improved
A further three skill areas improved on average overall by a score of more than 0.5, including:
• Develop a testable hypothesis for a research problem;
• Write a policy brief (given that you have knowledge of the subject of the brief); and
• Investigate perceptions held by people on a particular subject
There were a number of clear improvements in mode scores, including:
• Work in Excel;
• Conduct and write a literature review on the role of SMEs in the livestock sector
in Vietnam;
• Develop a testable hypothesis for a research problem;
• Write a policy brief (given that you have knowledge of the subject of the brief);
• Collect data to estimate a demand function for a product; and
• Design a survey to map/describe a supply chain
Trang 6Table 4 Comparisons of average and model baseline and end-or-project rating scores
for capacity to perform 15 specified tasks (Scale: 1 = very confident, 2 = confident, 3
= somewhat confident, 4 = a little confident, 5 = not confident)
Avg/Mode Rating Score
1 Work in Excel 1.90 (1,2) 1.44 (1)
2 Use Solver in Excel 2.90 (2) 2.69 (2)
3 Investigate the supply chain for a product 3.21 (3) 2.38 (3)
4 Conduct and write a literature review on the role of SMEs in
the livestock sector in Vietnam 2.52 (3) 2.38 (2)
5 Develop a testable hypothesis for a research problem 3.10 (3) 2.56 (2)
6 Write a policy brief (given that you have knowledge of the
subject of the brief) 2.69 (2,3) 2.13 (2)
7 Collect data to estimate a production function for a product 3.07 (2) 2.69 (2)
8 Collect data to estimate a demand function for a product 3.17 (3) 2.88 (2)
9 Design a survey to map/describe a supply chain 3.10 (3) 2.13 (2)
10 Build a mathematical model to describe the livestock feed
supply chain 3.66 (3,4) 3.31 (4)
11 Produce descriptive statistics from survey data 2.69 (2) 1.81 (2)
12 Investigate perceptions held by people on a particular
13 Estimate the competitiveness of a firm operating in the
livestock feed sector 3.14 (3) 2.44 (3)
14 Design a database in Access 2.69 (2)
15 Use Stata for data analysis 2.25 (2)
Note: Mode scores are shown in (in parentheses)
a Survey sample for the Baseline measure was 29
b Survey sample for the End-of-project measure was 16
3.2 Comparisons based on respondents’ level of involvement in
the project
As for the earlier questions about knowledge, a further comparison was made between
respondents with different levels of involvement in the project Scores for those
respondents who nominated themselves as either “very involved”, “relatively
involved” or “somewhat involved” were compared with scores from respondents who
said they were either “a little involved” or “not involved” The sample of 16 was
evenly split between these two categories The results are shown in Table 5
Taking the average ratings, only in four areas were skills rated better by those
involved in the project compared to those not involved in the project These were:
• Design a survey to map/describe a supply chain;
• Produce descriptive statistics from survey data;
• Design a database in Access; and
• Use Stata for data analysis
In all other skill areas, those not involved in the project rated themselves as more
highly skilled The four areas where those involved in the project rated themselves
Trang 7higher were areas where those most involved in the project (often more junior
researchers) had done a considerable amount of work
Table 5 Comparison of average and mode rating scores for capacity to perform 15
specified tasks: baseline results and end-of-project results for those respondents
involved and non-involved in the project (Scale: 1 = very confident, 2 = confident, 3
= somewhat confident, 4 = a little confident, 5 = not confident):
Avg/Mode rating score
1 Work in Excel 1.90 (1,2) 1.50 (1,2) 1.38 (1)
2 Use Solver in Excel 2.90 (2) 2.75 (3) 2.63 (2)
3 Investigate the supply chain for a product 3.21 (3) 2.50 (3) 2.25 (3)
4 Conduct and write a literature review on the role of
SMEs in the livestock sector in Vietnam 2.52 (3) 2.38 (2) 2.38 (3)
5 Develop a testable hypothesis for a research problem 3.10 (3) 2.63 (2) 2.50 (2,3)
6 Write a policy brief (given that you have knowledge of
the subject of the brief) 2.69 (2,3) 2.63 (2) 2.00 (2)
7 Collect data to estimate a production function for a
product 3.07 (2) 2.88 (2,3) 2.50 (2)
8 Collect data to estimate a demand function for a
product 3.17 (3) 3.13 (3,4) 2.63 (2)
9 Design a survey to map/describe a supply chain 3.10 (3) 2.00 (2) 2.25 (2)
10 Build a mathematical model to describe the livestock
feed supply chain 3.66 (3,4) 3.75 (4) 3.00 (2)
11 Produce descriptive statistics from survey data 2.69 (2) 1.50 (1,2) 2.13 (2)
12 Investigate perceptions held by people on a
particular subject 2.62 (2) 2.00 (2) 2.00 (2)
13 Estimate the competitiveness of a firm operating in
the livestock feed sector 3.14 (3) 2.38 (3) 2.38 (2)
14 Design a database in Access 2.63 (2) 2.75 (1,2,3)
15 Use Stata for data analysis 1.75 (2) 2.75 (3)
Note: Mode scores are shown in (in parentheses)
a Survey sample for the Baseline measure was 29
b Survey sample for the End-of-project Involved (End-I)measure was 8
c Survey sample for the End-of-project Not Involved (End-NI) measure was 8
There was also some evidence of improvement in mode scores for those more
involved in the project, compared to those less involved These were for the tasks
described by:
• Conduct and write a literature review on the role of SMEs in the livestock sector
in Vietnam;
• Develop a testable hypothesis for a research problem;
• Produce descriptive statistics from survey data; and
• Use Stata for data analysis
We would consider the results for these skills, which were an integral part of the
project work, to show capacity improvement directly related to the CARD project
Trang 84 Changes in attitudes
In Section C of the survey, respondents were asked to indicate their level of agreement with 18 statements Statements in Section C were designed to test whether respondents’ attitudes to various policy aspects of the livestock feed sector changed during the course of the project In a sense there are no right or wrong answers to these statements, but we would hope that the CARD project research would encourage
a more market-oriented policy perspective The average rating and mode scores for the baseline and end-of-project surveys are shown in Tables 6 and 7 The rating scale used was: 1 = strongly agree, 2 = agree, 3 = undecided or unsure, 4 = disagree, 5 = strongly disagree
4.1 Overall comparisons of baseline and end-of-project results
Overall comparisons between baseline and end-of-project results are shown in Table
6 Attitudes are often difficult to change, but some evidence of what we would consider positive attitude changes were evident Attitudes to three statements changed
by an average overall rating score of more than 0.5 These were:
• SMEs are unlikely to be competitive in the livestock feed sector In the
end-of-project assessment, respondents were more likely to disagree with this statement: i.e they were more positive that SMEs could be competitive This is consistent with the findings from the project, which show that medium-size mills are competitive, and that there is scope to supply niche markets and small landholders The breakdown of the scores is shown in Figure 1
• SMEs in the livestock feed sector in Vietnam are uncompetitive because they lack modern technology In the end-of-project assessment, respondents were more
likely to agree with this statement This is consistent with the findings from the project, which highlighted the technology differences between foreign-owned and domestic firms The breakdown of the scores is shown in Figure 2
• Farmers do not pay enough attention to the quality of processed animal feeds In
the end-of-project assessment, respondents were more likely to disagree with this statement This is consistent with the findings from the project, which indicated that farmers were very concerned about the quality of industrial feeds, and purchased foreign brand feeds because they perceived it was of better quality than domestic brand feeds The breakdown of the scores is shown in Figure 3
Attitudes to a further five statements changed by an average overall score of more than 0.3 These were:
• Vietnam agriculture should focus on growing the raw materials for processed livestock feed rather than importing them Respondents in the end-of-project
survey agreed more strongly with this statement Although the research work was unable to find significant price differences between domestic and imported raw materials, there is still a very strong feeling amongst Vietnamese researchers and policy makers that focus should be on improving the production of raw materials This is a difficult attitude to counter, and it has been exacerbated by the large increase in imported raw materials in recent years
• SMEs in the livestock feed sector in Vietnam are uncompetitive because they tend
to operate at a small scale Respondents in the end-of-project survey agreed more
strongly with this statement This is consistent with the project findings that indicated that, on average, small feed mills producing less than 10,000 tonnes output/year did not make a profit in 2007
Trang 9• There are not enough distributors of livestock feed in Vietnam Respondents in the
end-of-project survey disagreed more with this statement The research work did
not make any conclusions on the number of livestock feed distributors, but in the
areas where the work took place there was evidence of many distributors in
operation
Table 6 Comparisons of average and mode baseline and end-or-project rating scores
on level of agreement with 18 policy perspectives Scale is: strongly agree (=1), agree
(=2), undecided or unsure (=3), disagree (=4), or strongly disagree (=5)
Average/Mode Score
1 SMEs are unlikely to be competitive in the livestock feed sector 3.45 (4) 4.00 (4)
2 The government of Vietnam should support SMEs in the
livestock feed sector 2.17 (2) 1.94 (2)
3 Vietnam would be better off importing processed livestock feed
rather than manufacturing it 3.83 (4) 3.94 (4)
4 Vietnam agriculture should focus on growing the raw materials
for processed livestock feed rather than importing them 2.55 (2,3) 2.19 (2)
5 SMEs in the livestock feed sector in Vietnam are uncompetitive
because they tend to operate at a small scale 3.00 (2,3) 2.63 (2)
6 SMEs in the livestock feed sector in Vietnam are uncompetitive
because they lack modern technology 2.86 (3) 2.25 (2)
7 SMEs in the livestock feed sector are important for providing
employment for rural people 2.24 (2) 2.25 (2)
8 Foreign-owned companies operating in the livestock feed sector
get preferential treatment by the GoV 2.69 (3) 2.44 (3)
9 Large foreign-owned and joint venture companies operating in
the livestock feed sector in Vietnam provide good employment
opportunities for rural people 2.62 (2) 2.38 (2)
10 There are too many livestock feed companies operating in
11 It is advantageous if livestock feed companies can provide
additional services to growers – e.g nutrition advice, veterinary
services, grower contracts to purchase products 1.83 (2) 1.63 (2)
12 There are not enough distributors of livestock feed in Vietnam 3.45 (3) 3.81 (4)
13 Vietnamese SMEs operating in the livestock feed sector have
the capacity to adapt to changing market conditions 2.97 (3) 2.88 (3)
14 The quality of processed livestock feeds in Vietnam is generally
good 3.03 (3,4) 2.69 (2)
15 The quality of processed livestock feeds from SMEs in Vietnam
is not as good as that from larger foreign-owned companies 2.45 (2) 2.25 (2)
16 Farmers do not pay enough attention to the quality of processed
animal feeds 3.14 (3) 3.94 (4)
17 The production of livestock feeds in Vietnam needs better
regulation to ensure quality 1.62 (2) 1.50 (2)
18 Research done by CAP can influence government policy in the
livestock feed sector 2.45 (2) 2.06 (2)
Note: Mode scores are shown in (in parentheses)
a Survey sample for the Baseline measure was 29
b Survey sample for the End-of-project measure was 16
Trang 1010%
20%
30%
40%
50%
60%
70%
80%
Strongly
agree
Agree Undecided Disagree Strongly
disagree
Baseline Project end
Figure 1 Breakdown of respondents’ scores comparing baseline and end-of-project
assessment for the statement “SMEs are unlikely to be competitive in the livestock feed sector.”
0%
10%
20%
30%
40%
50%
60%
70%
Strongly
agree
Agree Undecided Disagree Strongly
disagree
Project end
Figure 2 Breakdown of respondents’ scores comparing baseline and end-of-project
assessment for the statement “SMEs in the livestock feed sector in Vietnam are uncompetitive because they lack modern technology.”
0%
10%
20%
30%
40%
50%
60%
Strongly
agree
Agree Undecided Disagree Strongly
disagree
Baseline Project end
Figure 3 Breakdown of respondents’ scores comparing baseline and end-of-project
assessment for the statement “Farmers do not pay enough attention to the quality of
processed animal feeds.”