Discussão:Brazil National Program consultancy process and upcoming visit in January/Review

Questions for reflections

1. Was the process appropriately transparent? If not, how could the process have been more transparent?
2. Is there a better way to involve the two representatives of the community in the interviewing process?
3. Did the on-wiki task for the finalists display their skills in community building and development?

Feel free to reflect on these, or any other question as well :) Jwild (discussão) 17h36min de 31 de julho de 2012 (UTC)

Feedback as selected candidate

I think it's high time I transparently share some impressions, feelings and opinions developed during and after the process with the community as a whole (I have already shared with a couple of people in informal conversations):

1. I really enjoyed the fact that I could have more contact with community members during the process. That was very important to understand a bit more about the community, have a feeling of what was to come and make sure that was precisely what we wanted and were keen to (or not).
2. That was very challenging to combine my busy schedule in my previous job and going through the assessment that seemed to require a lot of attention, especially because some extra activities (like IRC chat, new questions on specific issues and action in Wikipedia) were proposed throughout the week (requiring huge changes on my initial planning for developing the initial proposed tasks).
3. That was hard to deal with such a huge variety of views and opinions, in one single week, but it was very important to find out there was no "community single opinion". And that was very hard to keep quiet when I started seeing people's opinion about myself (especially because I felt sometimes people were giving opinion about the final page we built while the main criteria was supposed to be interaction. But also because I felt some people really got me wrong). But I managed to accept it all without reacting for the good of the process ;-)
4. Having seen everybody's opinions about all the candidates certainly impacted my first months working, making me realy walk on eggshells and making me less spontaneous and open than I usually am (like I'm being now). I'm sure transparency is important, but being aware of detailed personal opinions from people that knew you for one week online might not always help so much to start a relationship ;-). I'm not sure what is the best solution and balance, but I felt it was important to share it at this point.

--Oona (WMF) (discussão) 23h58min de 31 de julho de 2012 (UTC)

Off-topic

How did Jessie made these beautiful blue tables? --201.52.137.116 05h02min de 3 de agosto de 2012 (UTC)

magia! Jwild (discussão) 17h40min de 6 de agosto de 2012 (UTC)

community involvement

Community involvement: there was only a small group of Wikimedians (5-8) who really participated, resulting in some slower conversations on-wiki as well as one-sided arguments. It is important to get more engagement. I agree that is important to get more engagement however I think a group of 5 volunteers engaged in such activity is huge. In fact, if more volunteers get involved probably it would take more time to discuss anything. So, a few group discussing can't result in a slower conversation but the opposite. I know we should take care about anonymity of candidates but doing it in a open wiki was really important to others take a look about what was going on and maybe help with a specific point. I'm not aware of any problem about this process and I hope we could help WMF to hire others by the same way. OTAVIO1981 (discussão) 15h57min de 3 de agosto de 2012 (UTC)

Thanks for the thoughts, Otavio! That's a good point about the number of volunteers: I think we are really lucky we had a small group who WERE engaged - I think the main problem was that the candidates weren't able to get a real feel for the community through such a small number. But then again: maybe that *IS* the real feel of the community: a few very vocal and the vast majority not so vocal! I think it would be better with 10-15 active participants, personally, but I will change the language so it is not so definitive. If anyone else has thoughts on this, they would also be great to hear. Jwild (discussão) 17h45min de 6 de agosto de 2012 (UTC)

The thing that went terribly wrong but accidentally got silently fixed

Ni!

Ok, so one thing went wrong in the selection process in Brazil, but was left unwritten. It should no longer matter by now, so it is time to spit it out ;)

The process involved two volunteers who were selected by the community to be present and participate in the second of the three stages of the selection process, the stage which was not open to the public.

The intention being that the process should ideally be completely open, but in those moments where we agreed it was necessary to have confidentiality for the candidates' sake, those two volunteers would be present to oversee and participate in the condition of volunteers assigned by the community.

During that second phase, the process went well as planned and they did a good job.

However, after the second stage was over and the third stage began, they failed to understand that their assignment was related to the conditions of the second stage and was therefore concluded, even if occasionally the WMF team could contact them in private to discuss things that had to remain confidential.

Instead, they kept seeing themselves as above the community in the third stage of the process, which was decisive and completely public, and as a result, the WMF, which here also failed in their agreement with the community, started to see them as in a much more important position than the rest of the community during the third stage as well.

The candidates competed as equals in the third stage, but when the community evaluations of this last and definitive stage of selection were considered by the WMF, the community's opinion was completely overruled by the personal opinions of those two volunteers, who actively communicated their points of view in private to the WMF, and pushed the candidate of their preference.

And the only reason we even know this happened is because one of them - and I say this as a great and admirable quality he has - talks too much.

As a result of those actions, the candidate chosen was not the one publicly appointed as the best, not even second best.

At the time, I discussed this with a few other volunteers who had noticed the problem and were considering what to do, and we all agreed this was very bad. I cannot emphasize how very bad we thought this was. However, the reason we did not come forward with it is because it would be unfair with the selected candidate for the reason explained below.

It so happened that, all things considered, the process was not unfair to any candidate. Had those two volunteers simply participated as everybody else in publicly voicing their opinions during the third stage of the process, as they should have, the overall preferences among the top three candidates, now including the one which was selected, would have been very balanced.

In other words, had those two behaved as they should have, the chosen candidate would have been equally favoured as the other ones - something that does not show at all in the records since they did not publish their opinions like everybody else.

I'm going to repeat myself now, for the sake of being sure I'm understood.

Had we not heard, by accident, from one of those two volunteers about what happened, several volunteers would have created a lot of trouble given the official result. Enough trouble that the whole process would have had to be invalidated. I know of two other people besides me who would have put all their energies into invalidating this process had that volunteer not leaked in time that they both had pushed the prevailing candidate, and that the WMF actually wanted to pick one of the candidates the community publicly selected.

And again, the only reason we tolerated this was that those two are volunteers in the community and thus, had they presented their opinions publicly, everything would have been fine; and, under these circumstances, we did not want to be unfair to any of the candidates by invalidating an ultimately fair process they had put a lot of energy into.

Well, I'm repeating myself. I hope this helps the Indian community and the WMF to not make the same mistake.

As a related suggestion for improvement: I still don't like that the selection process involves any confidentiality at all. And the process in Brazil only convinced me how harmful it can be and how unnecessary it truly is. Many of the top candidates did not need that confidentiality, and I would much rather have a completely transparent process and narrow my choice of good candidates than have to deal with all the inherent costs and perils of keeping secrets within the community, let alone from it.

Hugs,

--Solstag (discussão) 02h41min de 4 de agosto de 2012 (UTC)

[...] "I know of two other people besides me who would have put all their energies into invalidating this process had that volunteer not leaked in time that they both had pushed the prevailing candidate, and that the WMF actually wanted to pick one of the candidates the community publicly selected." As explained bellow on the process being cumulative, I don't believe the bold part of this quote is true and maybe there is some misunderstanding here. --Everton137 (discussão) 16h41min de 4 de agosto de 2012 (UTC)

Tom, you told me that yourself in those very words, that the WMF team was prone to following the public comments from the community and you two made them change their minds and stick with your candidate. In any case, the WMFs position only clears them from criticism about this one point, it does not change any of the relevant facts about the problem that occurred. --Solstag (discussão) 17h04min de 4 de agosto de 2012 (UTC)

So I think there is a misunderstanding on what I told and I would like to understand where does it come from. There was a lot of agreement between me, Castelo and Barry about the candidates, as I always said, even in the meet up with the community at Centro Cultural São Paulo in the Saturday after the interviews. Yes, I and Castelo believe the choosen candidate performed better on the overall evaluation (interview + curriculum + wiki task) for this job position - in my opinion, some good evaluations by the community was completly right and I fully agree with a lot of them, but for another job position. What I remember of having said is that one candidate, which I think was not as good as the other two I liked best in the whole process (that is, Lepton and Photon), was well evaluated by WMF because of the general positive public comment. But I cannot remember of having said that "WMF actually wanted to pick up one of the community publicly selected" with precise words. I never understood that this happened. And I even said to you that I would bring two candidates to San Francisco for the final interviews. --Everton137 (discussão) 17h41min de 4 de agosto de 2012 (UTC)

Well, in that case, I take it back that the foundation is clear from criticism on this one point. --Solstag (discussão) 19h04min de 4 de agosto de 2012 (UTC)

May I add my two cents on this? I think this could have been much softened (or even solved) if, as Jessie pointed out, there had been a consensual position addressed by the community based on previously established criteria (even if this collective opinion pointed out more than one candidate for the final interviews if they were not able to come up with one single name. And if the community wanted to add or change criteria, that could have been established in advance). I think that the lack of consensus in this case weakened a bit the personal assessments, and may also have prevented some volunteer who knew candidates from feeling comfortable for sharing impressions that could have influenced final decision. In the end, the result is that WMF had the chance to take into consideration volunteers comments, but wasn't pushed to follow community's final opinion. I think that when a debate goes on and a collective opinion is made, it gets much stronger and the arguments for this or that choice, much more well developed. If possible, I think that would be valuable for India to really try a consensus (again, even if points out more than one single candidate), if this is possible--Oona (WMF) (discussão) 10h55min de 6 de agosto de 2012 (UTC)

Trying to achieve a consensus on pre established criteria would help a lot and would improve the process. But we need to be aware this can take a longer time and, depending on the number of volunteers that can get involved, there won't be a consensus. WMF needs to have deadlines, volunteers not. In my case, I just haven't add my opinion, as explained bellow, because of the lack of time and I left to evaluate only after all my friends. --Everton137 (discussão) 11h39min de 6 de agosto de 2012 (UTC)

+1 to Oona's point -- I think the format of individual opinions made it more difficult to decide what opinion was "the community's." Maybe the role of the two volunteers who are to present the opinions of the community could entail taking any and all personal decisions listed from a talk page and attempting to compile the consensus view on the main page. That way, if anything was totally off, it could be corrected publicly. Also, this would give these selected volunteers the opportunity to wait in presenting their opinion publicly until the others have had a chance to make their independent assessments without the additional information of interviews/resumes (as Everton indicated was his rationale in delaying participation). Jwild (discussão) 18h19min de 6 de agosto de 2012 (UTC)

thanks for the long, detailed feedback, Solstag. One thing I think is important to mention is that there was NOT a community consensus on this, and even though there were viewpoints presented by only four individuals, these were not consistent and also did not all follow the criteria initially set-up in terms of evaluation. So, I don't agree with the summary that there were two candidates "appointed as the best...[and] second best." (I do agree with your overall point, but I think this is an important distinction to point out regarding the last process.) The important thing about community is to help filter and weight feedback based on its rationale and relevance. So - that doesn't completely negate your over-arching point of needing a consensus (I just don't think it should be a voting process), and I propose, then, that the role of the two selected volunteers should entail providing feedback on individual opinions that perhaps are not matching the established criteria, in addition to compiling all the feedback into a consensus statement. Jwild (discussão) 18h19min de 6 de agosto de 2012 (UTC)

---

Excuse me, can I interrupt?

I agree with much of the Abdo said. And I think what you should look for, in the text, is not whether the process was wrong or right, but the vision that many of the community have/ had, and how it can / could hinder the progress of the candidate, once elected. One more important thing, the process must add value, the lack of transparency in the process was very bad, and will be worse if it occurs again; and for Brazil, if the next candidate (Community Facilitator) did not show at least who he is on Wikipedia, you will have many problems.

Stop trying to defend yourselves, and start to understand what we feel, where you went wrong, how can you improve the process, and how not to go over the opinion of the community.

And sorry Jessie, but four people in this community is a significant number, and in this case, there were only two of the Wikimedia Brasil, and in this case the number was low. And because the lack of transparency, we do not send two volunteers to talk with the candidates, but one, because the other was an employee of you. And I felt betrayed by this attitude...

And it may not have been a consensus, but if you ask the community what they thought of us stop all our activities to perform an activity with you, and our opinion does not have relevance in the decision to you, there will be a consensus that was very bad for our relationship.

Rodrigo Tetsuo Argenton (discussão) 08h25min de 7 de agosto de 2012 (UTC)

Thanks for taking the time to provide some feedback, Rodrigo! Everton made a comment below about his employment. I am curious: if the Brazil community felt this way about him as a volunteer, why did you all select him to begin with? Jwild (discussão) 21h15min de 8 de agosto de 2012 (UTC)

I do not know the community opinion on Everton case, I know that the community was upset when the opinions were not heard. I particularly felt betrayed. There are two situations...

And I do not know what was the reason for he being chosen. Rodrigo Tetsuo Argenton (discussão) 21h47min de 8 de agosto de 2012 (UTC)

I can see why this seems like a conflict of interest. At the same time, it does seem like a potential conflict the community should have thought of if they automatically do not trust someone who works with WMF to represent them. (For your reference, I just looked it up and the selection process was held on this wiki, where about 8 of you voted.) Jwild (discussão) 22h03min de 8 de agosto de 2012 (UTC)

Hi Jessie! Rodrigo meant the community was upset about feeling their opinions were not heard about the candidates, not about Everton. About him, he was just being hired and everybody was OK with that, also, some people hired private investigators to track him ;) --Solstag (discussão) 05h38min de 9 de agosto de 2012 (UTC)

Again we are off track, I think we should create selection processes that involve the community, and ensure the voice of the community. Also must be a transparent process. The end! Rodrigo Tetsuo Argenton (discussão) 14h37min de 9 de agosto de 2012 (UTC)

Some comments on the community evaluation and the process

First, repeating here the links to the task proposed to the candidates

• Tarefa para a Consultoria ao Programa Nacional do Brasil (see the talk page as well and I hope a google translation can give at least a rough idea on how it was for those interested)

and the community evaluation (not those selected to participate of the interviews - explanation bellow)

• Tarefa para a Consultoria ao Programa Nacional do Brasil/avaliação

I know the links were given in the main page of this good review, but just highlighting them.

On not publicizing my opinion together with the community evaluation: first of all, I was one of the volunteers selected by the community* and I believe I should have publicized my evaluation to avoid the feeling of the process not being transparent and of people not assuming good faith, but unfortunately it wasn't and I must add some comments about this. Since I followed the interviews led by Barry, I had too much extra information about all candidates, so my goal was to evaluate them only after my Wikimedia friends, having in mind I could influence the evaluation if I was one of the first to do it. (*Note: to be more precise, by community here I consider that small number of people involved with Wikimedia Brasil activities, where most volunteers more dedicated to Wikipedia project are not too much involved.)

After the community evaluation, there were deadlines for discussing about the results and I simply haven't had time to publicize my opinion. So after the discussions to select the best candidate(s) based on 1. the candidates performance during the interviews, 2. the candidates curriculum, and 3. the evaluation by the community on the candidates performance during the tasks, we have had a choice through a cumulative process. It is completely understandable to see opinions like "WMF started to see them as in a much more important position than the rest of the community" and "the community's opinion was completely overruled by the personal opinions of those two volunteers", and I must say I have a great responsibility for that. If I and Castelo, the other choosen volunteer, added our evaluation publicaly, this feeling could have been dimished and wouldn't make people think that they were just "personal opinion", but opinions based on informations we had access to.

That said, I need add every single evaluation was fully considered and my friends evaluation were taken seriously. I was even encouraging some of my volunteer friends to evaluate, but only only four have done so and I am sure much more people have at least read what candidates were writing. Everybody could participate of the process and evaluate the candidates performance, but what happened is very well explained by Otavio above and I agree with him we should have more engagement of all wikimedians. So here is an important question that the answer could help to improve the process: why a lot of people from community (or communities) haven't felt part of the process? If more people participated, the discussion would become longer and the candidates could have much more trouble, but this could also be good way to evaluate which candidates can perceive what is essential in this sea of parallel conversations. (Guess about the non participation: Maybe the volunteers were just not interested or couldn't understand what was being done? Or, as I've been observing through all my life, some people tend to just show up with critics and ideas on how it could be only after decisions are taken or actions are made, and this very basic fact of life likely will never change, Greek and Trojans cannot be both satisfied.)

On the interviews and confidentiality of candidates: I think it is important to tell that after each interview, Barry, Castelo and I have shared our impressions about each candidate and interestingly almost all our opinions on the goods and odds of the candidates were very similar. We also had the opportunity to read their curriculums beforehand and I and Castelo could make additional questions to all candidates, beyond the ones proposed by Barry. According to a volunteer who knew all candidates in this last phase and could speak to some of them (all? please, confirm), I think it is also worth mention how well these interviews were perceived.

Regarding the confidentiality, if you live in the real world, I think it is out of question not being like this. It is completly fair a candidate to have another job position (and they were really good candidates, which makes more natural that they already have important roles inother places) and disclosing if he or she is competing for another one can put him or her in a embarrassed situation, to say the least. And this is the case of at least one candidate that didn't want to have this public at all. If this is the situation of one candidate, this must be a general rule. During the interviews there were, of course, informations shared by the candidates such as their expectations about the job position, their career perspectives, among other things, which, well, can not be disclosed and that is not possible to know through the task. So the community have to trust the other volunteers they have selected to have access to this information.

I will repeat that this kind of process is cumulative. The community(?) opinion, or, better saying, the 4 opinions publicaly shared, have made me confirm some impressions I've formed through the interviews and helped to change some of them as well. And this was considered when I had to discuss the candidates that would go to San Francisco to be interviewed and also before the interview, although the short time to absorb all that (this can be also a problem, but we need have deadlines, so how to adequate the process to that?).

One thing I began to wonder only after sometime of this process is that I was already working for Wikimedia Foundation as a partial time contractor for the education program, so maybe I should not have participated of the interviews. Sure I still had my free time which I continued doing activities as a volunteer, but maybe having another volunteer would help the community to gain more trust about the process - and I bet the result would not change. Now after months working with the choosen candidate, I must tell an important characteristic of her I couldn't fully notice before: she has really assimilated what the community have said about her performance in this task and she works hard to try to improve it. Although the possible conflict of interest here, you can judge by yourself if you agree with me through the work she has been doing in the last months. --Everton137 (discussão) 16h26min de 4 de agosto de 2012 (UTC)

Adding one point that was also taken into account: Jessie has also interviewed the 4 candidates. And her assessment were also considered in the whole process. That is expected. She has worked since a long time with the Brazilian community and worked closely to Carolina Rossini, who provided advice as a consultant for Wikimedia Foundation for one year for the Brazil Catalyst Project, so her experience, obviously, was considered. --Everton137 (discussão) 11h47min de 6 de agosto de 2012 (UTC)

what should be the output of consensus?

We have mentioned "consensus" a lot in these discussions, and I'm wondering what the output of reaching consensus should be. Should it be simply a ranked table of the different candidates? OR should it be simply a table of the candidates and their pros/cons as demonstrated by the online assignment? The former is easier to digest as a statement from the community, but the latter is more representative of the fact that the results of the online assignment are just one piece of the information going into the final decision. Jwild (discussão) 18h23min de 6 de agosto de 2012 (UTC)

Good point Jessie! I'm not sure this should be a task for the two volunteers participating in the previous stage. Instead, this should be a task for the WMF team because, since it is them making the final decision, they should be the ones expressing what they understood from the volunteer's comments and what they feel was more or less important, so the volunteers can respond and fix what was wrong and what was missing in the decision maker's summarized mental picture. The two volunteers who were assigned to participate in the second stage could, instead, be responsible for translating the opinions expressed in native language. I also much prefer a written testimonial/impression from the decision maker than a table or any pseudo-objective summary; on the other hand, the decision maker should feel free to also publish whatever seems more natural to the way he is thinking about the matter, be it tables, bullets or his 5 year old kid's hand drawn caricature of the candidates. --Solstag (discussão) 02h43min de 7 de agosto de 2012 (UTC)

Definitely the hand caricatures. No other option is acceptable :P

But seriously: I actually think it would be better to have the volunteers leading the discussion in community consensus rather than WMF. This way, that feedback is consolidated and curated by the community itself, rather than WMF trying to go in and extrapolate from a set of comments. Once the community consensus piece is given, then I think WMF should be able to take that into account. I'm not sure who you mean by "the decision maker" -- do you mean WMF? So, at the end of the process they should write up a testimonial/impression? Jwild (discussão) 21h14min de 8 de agosto de 2012 (UTC)

What I am worried about is that what we write down as consensus and what the WMF team (the decision makers I mentioned) actually understands from it can be two quite different things. Only if the WMF team members themselves expose their understanding of what the community is saying and how they interpret it, can the community reply to that by fixing any misunderstandings or misfocusing. I don't believe an information rich consensus can flow through separate groups without dialogue, and just writing down a final consensus to be taken into account is not dialogue. That should not stop the community members from discussing and trying to summarize some ideas among themselves, but forcing them to reach a final consensus that is not a deliberation may just lead the community to be too broad in a way that the WMF gets to pick whoever they feel good about without any window for criticism. In any case, this is already at a level of detail that won't make so much of a difference, but I do worry that keeping the same volunteers in privileged positions over and over is not good, and we should replace the assigned volunteers between the different stages as the purposes are different, if only to help avoid the kind of problems that came up. --Solstag (discussão) 05h29min de 9 de agosto de 2012 (UTC)