Welcome to our very first Indigenous Initiatives interview series exploring decolonizing approaches to evaluation! Across this series, we’ll be speaking with colleagues from across the University of British Columbia who hold different roles and responsibilities, but who are all deeply engaged in research and evaluation work. Together, they help shape how knowledge is gathered, interpreted, and mobilized within the institution and beyond. In this first conversation, Carissa Block and I had the pleasure of sitting down with Wendy Bond and Asma-na-hi Antoine from the Office of Research Ethics, and Sam Filipenko from the Indigenous Research Support Initiative (IRSI). They reflect on the trends and shifts they’ve observed in research and evaluation over the years and explore how evaluation practices might better respond to the priorities and needs of communities. Our lively discussion also reveals some of the common barriers to doing evaluation in ways that align with decolonial approaches within an institutional setting, and some hopes and dreams for what the future of evaluation could look like.
Wendy Bond: Has been working in the Office of Research Ethics at UBC since 2013. For the past few years, she has been leading projects to implement UBC’s Indigenous Strategic Plan into operations at the Behavioural Research Ethics Board. The most recent is a pilot alternative Indigenous-focused ethics application, which is mentioned in the conversation.
Asma-na-hi Antoine is from the Toquaht Nation and has been a visitor to the unceded and occupied traditional Lands of the xʷməθkʷəy̓əm (Musqueam), Sḵwx̱wú7mesh Úxwumixw (Squamish), and səl̓ilw̓ətaʔɬ (Tsleil-Waututh) ancestors and peoples. She is the Indigenous Research Project Officer in the Office of Research Ethics, working alongside Wendy Bond, supporting and implementing research ethics projects, including the pilot alternative Indigenous-focused ethics application.
Sam Filipenko: Sam (he/him) is a white settler of English and Ukrainian ancestry, born and raised on the ancestral, unceded, and occupied territory of the Musqueam people. He works with the UBC Indigenous Research Support Initiative, supporting community-led and collaborative research between UBC and Indigenous partners. His work is grounded in a commitment to upholding the knowledges, protocols, and sovereignty of First Nations, Inuit, and Métis communities, and he’s been taught that decolonizing academic research practices requires each of us to hold a piece of the puzzle.
This interview has been edited for length and clarity.
The first question is around this idea of evaluation and research. Can you tell us how the Office of Research Ethics differentiates between them and what your take is on these two terms?
Wendy: So, we do recognize that there are types of evaluation that may use research methods, but that doesn’t make it research. So that’s the first distinction that we try and make for people. If the purpose of your project, let’s say, is to evaluate an existing program or a piece of technology, those types of projects do not require ethics review, If your only intent is evaluation (quality assurance – quality improvement), you do not need research ethics. You could be using surveys or interviews, or discussion groups. That doesn’t make it research. The other piece that that often gets people caught up is, “but I intend to publish my results”. Publishing results is also not a criteria for whether it’s research or not. We have that conversation with researchers and then they look at what they’re doing just to make sure that, yes, indeed, they’re doing evaluation exclusively, not research.
Paulina: But evaluating a program could be a part of someone’s research.
Wendy: It could be a part of it, yes. So, for example, if you were evaluating three different programs and then looking at the similarities and differences between them and creating new knowledge out of that, then, yes, that’s verging into research. And that’s why it’s really important to understand that the methods that you use aren’t what makes it research. It’s your intent. It’s really important for people to have that conversation before they start. They need to think about what the potential outputs of their project might be. Sam, do you have any interaction with that question when you’re talking to community partners?
Sam: Yeah. When I’m talking with community partners, I would say that the distinction between evaluation and research is not as relevant to their needs as partners.
Wendy: Yeah, because the relationship is still important regardless of what the outcomes are.
Have you noticed any shifts in the kind of projects that people are evaluating or researching these days, either in focus area or in their approach?
Wendy: The biggest change is that we are pushing back on researchers who have not established a relationship with their research partners. We used to review those applications and give them conditional approval. Pending them establishing that relationship, and that might result in changes to their application. It should result in changes. We don’t tend to accept them [for review] anymore. We say, you need to go and work on your partner relationship before you submit your ethics application. There’s also a little bit more clarity around the nuances of relationship building and what that constitutes within a research project. We used to have researchers say that they were doing community-based research because they were researching a community, and that’s not what we mean. Maybe there is more of a sophisticated understanding of those dynamics within a community where research is happening. I will say that I think the greater sophistication is going to have an impact on all behavioral research, not just on Indigenous focused research.
Asma-na-hi: I definitely think there’s a heightened consciousness about the need to build relations. I think over the years reviewing research projects, I’ve seen an increase in heightened awareness and a shift between “I’m going to go research on this Indigenous community” versus “I’m going to go research with the Indigenous community.” I think one requirement is that a researcher must write a proposal. You review the proposal; it is approved, and then it goes to ethics. I think about the relationship between the PI and the researcher, who would have connected on the topic, and then the departmental approval to also connect to it. So, there were a lot of people involved in a process. But where should we start to ensure that, by the time it reaches the ethics office, all gathered information is correct? And so, when the ethics application does get to us, we don’t have to send it back. Because at times, when an application has been sent back, researchers feel frustrated, throw their hands up in the air and say, “What? How do they get all the way down to the ethics application and not be informed a lot sooner in the process?”
Wendy: Yeah. We were having a conversation about this last week – there’s a lot of space for conversations around the design of the research to happen well before it comes into the ethics office, and even before funding is sought. I don’t think we have really figured out a way of supporting researchers in the conceptualizing stage. That’s where things can move forward at a great pace without someone asking “have you thought about X, Y, and Z?” To me, that’s one of the areas we need to focus on and figure out how we can support researchers and students. The other piece in the early stages we haven’t really addressed is [to the researcher] what are you doing after you’ve gotten your data, done your analysis and published your research paper; what is this relationship with the community going forward? Our pilot application asks about that, but I think for some researchers, it hasn’t come up. That’s an area where learning is needed. For us as well, because we have to ask the right questions in order to get that information from researchers; the ethics application does not currently concern itself with the future.
Sam: I totally agree with Asma-na-hi – I find that there’s more awareness of needing to do research in collaborative and relational ways. But I still notice there’s still a disconnect, like Wendy said, at the front end of projects, where researchers rush to prepare their study design without meaningful relationship building or community involvement. And it ends up being this lopsided situation where proposals can get formulated from a place of very little relationality, where it’s like, maybe the researcher had one conversation, or they might have attended one gathering, and then say in their proposals that, well, “we have a relationship”. And sometimes communities are then like, “no you don’t. Who are you?” So, I do think that we need to support researchers, but we also need to support communities to have the capacity to even engage in the study design process. Because there’s such a difference in time, energy, and resources to even consider research projects, and that difference in capacity ends up creating imbalances in partnerships and projects.
The Indigenous Strategic Plan specifically goal three, seeks to support research initiatives that are reciprocal, community lead, and legitimize Indigenous ways of knowing and promote Indigenous people’ self-determination. For researchers who are in the early stages of developing a project, what considerations would you recommend that they keep in mind before submitting an ethics application?
Wendy: It needs to start well before the ethics application is submitted. There’s the issue around funding applications and who gets the funding. Who is the funding on behalf of? Has the community or the population that is going to be affected by the research been involved in designing the research? And from a research ethics application perspective, it has to do with — I don’t know what the word would be – customizing your documentation to be appropriate for the community where you’re doing the research or on whose behalf you’re doing the research. And so, there’s often some time needed by the researcher to find out what the protocols are within that community — for asking questions, for inviting people to participate in a project. How do you get invited into the community in your capacity as a researcher? Who do you need to talk to, to ensure that you’re actually getting the appropriate approvals and consents in place before you start doing the research? That varies from community to community. We can’t tell the researchers the right way to go about conducting their research in a specific community. They need to go and do that work themselves and become informed about it. Otherwise, we have no assurances that what they’re doing is actually appropriate.
Asma-na-hi: I want to include three points I’ve shared with Wendy: the work we do supports the researcher and helps ensure they conduct themselves ethically. To remind them they are not just representing themselves, they’re representing the university, but more importantly, ensuring that their goal and intention are to protect the community and participants. As we discussed, there are gaps in the research process and misalignment among the funding application, the proposal, and the ethics application. And so recognizing some of those gaps in navigating through and supporting and finding answers on behalf of everyone who is sitting at the table, understands, and is all well-informed about the processes, even after a research project begins, then who do they call when they have a grievance or a complaint about the researcher? Because then, you know, just ensuring that all communication lines are open.
What barriers or challenges do evaluators face when trying to carry out rigorous ethical or decolonial evaluation projects?
Wendy: I think it would be interesting to ask the question around rigour and secondly, around ethics. I think the barriers would be different for each. I mean, all research should be rigourous, but I’m assuming that that’s about getting data that you can stand behind. What’s your experience?
Asma-na-hi: So many (laughs). Thinking about barriers does not mean understanding the colonial aspects you’re trying to decolonize. Defining even what that means. So, going back to your example earlier and saying, oh, you want to think about another way to collect data and decolonize that. And you mentioned surveys, but I view surveys with caution because if you have a classroom with 20 students and only two are Indigenous and they mark “Indigenous,” then those students would be targeted, and it would not be a confidential survey. And so, recognizing that some researchers may not understand the implications of surveys or may approach their work in a colonial way or process, is not “I’m doing the right thing”. Yes, kind of, but you’re also not thinking about the aspects or implications of participants.
Wendy: Is it a barrier or more just a challenge in moving forward? I’m not sure what barriers exist, because as a university researcher, they’re coming from a fairly privileged perspective. I mean, I think the barriers would be on the community side more than on that academic side.
Asma-na-hi: For sure, and you think about the financial process, not the funding, but like the processing to get funds. To give an honorarium to a community member. There’s a barrier because of an institutional process that lacks a respectful way of engagement between the researcher, the participants, the community, and the collection of information, and so on.
Wendy: Yea, I like the word, I think challenges resonates better for me than barriers, because if it’s an attitudinal challenge, it’s not a barrier unless they’re unwilling to address it. You know? Maybe one of the larger challenges, just very generally, is time. The academic schedule for delivering research is not conducive to community relationship building. And particularly for graduate students who are driven to complete their program; very often the scope of what they’re wanting to do just cannot be done within the context of a community involved research project. We do sometimes have to push back and say: take one of your objectives and maybe you’ll be able to do it, but you need to go back and do that work with a community before you even finalize your research proposal, let alone submit an ethics application. I think students feel that acutely, that limitation or the pressure of time. We may agree that Indigenous focused research has evolved over the last five years or so to understand the importance of doing that upfront work before finalizing a thesis proposal, but have expectations changed in the curriculum? I don’t know what a change would look like, but I think it would start by having the designers of those degree requirements accepting that the existing model does not work. If there’s a desire to have that kind of research done by graduate students, the requirements may need to change.
What hopes and dreams do you have for the future regarding research and evaluation?
Wendy: Asma-na-hi I think that you have a really good one…talking about the commitment of resources and time on the part of the university to make these things work. I think it’s been underestimated; you can’t do this work of transformation off the side of your desk. And it takes time to change culture, and institutional culture still needs change. That is a massive undertaking; publishing strategic goals is just the beginning.
Asma-na-hi: Very beginning. There definitely needs to be an institutional change to build a stronger understanding of research processes and policies. And one of the things I’ve said to Wendy and Sam is that I am- I’ve been working in academic institutions for a long time, and recognizing that Indigenous ways of thinking and being and Indigenous lenses don’t fit into a university policy structure. It needs to be a standalone entity and navigate through that process, and some universities and colleagues I know have started that process. We’re considering where we stand; there’s been significant improvement in Indigenous research and ethics. But there’s so much more that we can do collectively if we have the promise in the plan, then why are we not implementing it into the different departments that are necessary to get it done? And for sure, I understand funding is a parameter, but also not just within the university, but within nations as well, within Indigenous communities as well is that navigating and supporting their capacity building, whether it’s, you know, training their members to be able to understand, to support the work that they’re doing there, that reciprocal relationship that needs to be ongoing and continue. If you want to work with them, well, how are you going to build their capacity so that we, you know, get the work from them and with them.
Wendy: There’s a lot of [institutional assumptions] that are embedded in policy without us necessarily recognizing that they represent a certain attitude. One that we’ve been working with in the pilot project came out of our discussions about what kind of information we want a researcher to be able to provide when they’re submitting an ethics application. The concept of humility is not necessarily a value that is entrenched in academia. How do we ask a researcher to practice humility? We can ask questions that allow them to demonstrate their humility. We currently ask a TCPS2 driven question: “what are the benefits and risks to the research population?” Our new question is, “what are the benefits to you as a researcher? How are you benefitting from the privilege of doing your research within this community?” That’s just one example.
Sam: There is a big need for the institution to transform its research culture. And while I see bits and pieces of it shifting, like an increase in awareness and enthusiasm for conducting work with Indigenous community partners, there are still colonial research practices that are deeply embedded into policy and the way that people here operate. Like taking an extractive approach to data collection, or when research creates exclusive benefit to the researchers, to name a couple. In this way, when we describe the University as being a colonial institution, it’s not just a metaphor. It’s a place that has had its power built from these practices and policies over decades and decades.
Wendy: Sam, it is so important to show those examples; I do not talk about “colonial” anything. I don’t find the term in itself helpful in moving the conversation forward. Part of the work we did early on with developing the alternative ethics application was to consider, if somebody sees the BREB ethics application as being a colonial instrument, what does that actually mean? Question by question [in the ethics application], what about this is colonial? Coming to an understanding of that was far more valuable for me. It’s not just about bringing people into the circle, because that still doesn’t tell you what you need to change. Asma-na-hi was talking the other day about how we update, revise, or reconfigure our existing policies to be more conducive. She said, we need to start from scratch and that resonated for me. We just need to create an Indigenous policy and forget about trying to massage the other ones. The same thing happened with the ethics application: we cannot work around the edges of this application form to remove those unhelpful aspects. They’re embedded in every part of the application form, so start from scratch.
Sam: I completely agree; we don’t need to tinker around the edges of policies and practices. We can just open up whole new ways of doing things, which I think creates new spaces of abundance. This work can’t be based on scarcity and carving out things from the edges. We can just do entirely different things instead. We can create what serves the kinds of research that we know to be mutually beneficial and that researchers and communities want to do.