Research can be considered feminist when it is grounded in a set of theoretical traditions that privilege women’s issues and experiences(7,8). A key feature of feminist research is its focus on power imbalances, both in the subject that is studied and the relationships between the subject and the research. Feminist research practice includes the practice of reflexivity or positionality, both as a tool for producing knowledge and as an ethical method focused on subverting unequal relationships of power in knowledge production, considering our own position within the research as one of power, and always taking the vulnerability of the researched into account(9,10). An ethical commitment cannot happen without the step of reflexivity(11). Reflexivity is the deconstruction of knowledge production that works by addressing academia as a place for knowledge production, where social interactions and thus social constructions happen(12).
Gender mainstreaming can be used as a strategy for achieving equality by integrating gender analysis in all the phases of policies, actions, plans, and budgets(13).
Gender as a variable of analysis looks at social attributes and opportunities associated with being female and male, at the relationships between women and men and at the relations between women and between men(14,15). Moreover, as gender is generally understood as a social construct affected by power structures, different analytical concepts have been used to perfect the analysis of those power structures and its relations to gender. The most widely used concept is intersectionality. Intersectionality is used to map out the intersecting power relations in social relations by viewing categories such as race, class, gender, sexuality, nation, ability and ethnicity, among others, as interrelated and mutually shaping one another(16). As such, by understanding gender as a variable that is central to our research, we go beyond sex disaggregation and look into what the data conveys about attitudes, norms and gaps within AI crowdwork. Within the project, we conducted two survey pilots, taking care to prevent the exposure of the women identified as subjects of research (crowdworkers), to abide by ethical standards and fair pay, and to open up communication channels for them about their lived experiences beyond what previous literature on women crowdworkers has revealed.
Our team conducted two survey pilots in Spanish on the Toloka platform. Toloka is a global crowdsourcing company, founded in 2014 by Olga Megorskaya and integrated within the Yandex search engine. It is an enabling environment to support data-related processes(17). We chose the platform as it allowed us to filter workers geographically and thus to specifically analyse Latin American workers(18).
Additionally, we contacted Toloka’s Educational Program team, who expressed interest in supporting research activity through the platform. The first version of the survey was first answered by 6 people (three men and three women); the survey pilot was divided into specific sections, and most of the questions were either Likert-scale-type or open-ended questions. The time allocated for submitting the survey was 20 minutes for female and male crowdworkers who were interested in doing the task, and a filter for sex was used to get the answers of a group of women and men separately. Even though Toloka only permits binary sex disaggregation, our survey included an option for more inclusive gender identity. As we became employers (or “requesters”) on the platforms, we offered payment for the completion of the survey as if it was a task, calculated based on the minimum wage in the United States and the time to complete such a task(19). The payment to submit the survey was calculated as (minimum payment x fraction of an hour). For example, if we assumed a payment of USD 7.25 and the time to complete the survey was 20 minutes, the worker would receive an amount of USD 2.4 (7.25*0.333). To gain insights into how gender affected respondents’ work, we used different questions regarding widely recognised gendered factors such as time poverty(20), work-life balance, outside attitudes to the work done, care work(21) and direct opinions about whether they had considered gender a liability to their work. The second version of the survey was an iteration based on feedback about the need to include additional questions to better understand the characterisation of women working in crowdwork in Latin America, their motivations, needs, socioeconomic contexts, skills and hopes. The sample for the second survey included 57 women and three men.
There were four age groups represented in the sample, and 76% of respondents were 18 to 35 years old. The majority answered that their gender had little or no impact on their experience performing crowd work, while 17% believed it had some or an important impact on it.
As could be expected for the age group, most respondents had a professional degree or had at least started a university degree programme; 13% held a master's degree. Regarding their professional self-assessment, most of them knew their work strengths, work style and work environments and had a long-term vision for their career. They could balance their professional and personal goals and they wanted to develop new skills. Other respondents regarded themselves as being neutral in goal balancing, building a long-term vision for their careers and identifying work environments that interest them in their career search. Few of the respondents perceived that they did not know how to search for careers relevant to their interests and did not have a long-term vision for their careers.
They did not work in public spaces or libraries. They worked about two times per week from their own devices and at home, and spent about 20 minutes completing a common task, in some cases up to 40 minutes; the task they performed most was data labelling, webpage testing, and surveys (only 10% performed audio transcription tasks). Income per task was in the range of $0.01 to $0.16. 91% of respondents were engaged in other economic activities in addition to their work on Toloka. The most recurring reason for them to perform crowdwork tasks was to gain supplementary income. Additionally, a preference for working from home, and having schedule flexibility, were important factors. Few people considered it a recreational activity, and one respondent mentioned crowdwork as one of only a few opportunities for employment. Regarding the impact of the geopolitical situation of their country of origin, participants considered the main impacts to be economic. These economic impacts reduced their professional prospects and required them to engage in additional economic activity; 21% remained neutral.
Most importantly, the respondents shared an understanding of what features would be preferable in a tool. The explicability of tasks was important; although crowdworkers in Toloka can contact the requester, they find multimedia resources and tutorials useful to fully understand the tasks. Similarly, as 63% of respondents use search engines and translating applications as tools, prioritising language learning features is important. Regarding the perceived need to share experiences with other workers, the main goal of a communication feature would be to increase task completion, profitability and performance.
With the support of <A+> Alliance’s Feminist AI Research, we are currently developing a prototype for an AI-powered social connection and recommendation system in the form of a website specifically designed to assist crowdworkers in building a supportive community and developing their technical and soft skills.
The website is due to be accessible to the public in September 2023. We have started to explore conversational-agent systems based on our survey results, understanding the unique needs and challenges facing crowdworkers. The chatbot will be designed to match individuals with similar interests, goals and skill sets, enabling crowdworkers to form meaningful connections and giving them opportunities for skill-sharing and collaboration. The AI of a system that connects crowdworkers would likely involve a number of different technologies and techniques. One key component would be natural language processing (NLP), which would allow the system to understand and analyse the language used in the survey responses and profile information provided by users. This would enable the chatbot to understand the interests, skills and goals of each individual and match them with others who have similar characteristics.
Another important component would be machine learning algorithms, which would be used to analyse the data collected from the survey and continuously improve the system's ability to make accurate and relevant connections. These algorithms would be trained on the survey data, they would be able to identify patterns and relationships between different users, and they would have the capacity to make predictions about who would be the best match for a given individual.
A socio-technical intervention to leverage gender perspective and public policy analysis to design smart tools to empower the AI workforce in the Global South.
