1. Auditing Recommender Systems for bias towards Content Creators
Recommender systems significantly influence content creators' visibility and livelihoods, yet their design often prioritizes user metrics over creator needs. This can lead to perceptions of unfairness, a lack of control, and potential burnout within the creator economy. This project directly supports a larger research program aimed at identifying and mitigating biases towards creators and developing more equitable and transparent recommender systems. The student will assist in two critical early phases: auditing existing algorithms in different platforms using sockpuppet auditing and drafting a conceptual framework for creator-centric fairness and agency by analyzing the audit results, and existing fairness measures. This work is designed to refine definitions of fairness from the creators' perspective and to brainstorm tools and mechanisms that would provide them with meaningful agency. The primary goal of this project is to provide foundational support for understanding and operationalizing creator needs in recommender system design. Specific objectives include: Assisting in the synthesis of literature to contribute to Persona Definition: designs unique, automated bots (sock puppets) and assigns them highly specific "viewing" or "browsing" histories for creator-centric fairness and agency. Behavioral Simulation: Develop automated bots perform controlled actions—such as liking, sharing, or spending a specific amount of time on a video—to teach the platform's algorithm their supposed preferences. Data Logging: The audit system scrapes different feeds to see how content recommendations evolve, comparing unpersonalized and personalized feeds. This project will employ a sock puppet algorithm audit. The student will engage in: Define sockpoppet personas based on literature and diversity of Canadian creators and Audience addressing biases affecting under represented creators with focus on racial and linguistic discoverability Supporting implementation of sockpoppet audit on digital media platforms. Conduct Sockpuppet audit of platforms such as youtube and spotify under supervision. Assisting in literature review and documenting and analyzing audit data.
Research area, student roles & skills
Research area: Dr. Afsoon Soudi is an Assistant Professor at Toronto Metropolitan University and the Associate Director of Creative AI hub at The Creative School. She led machine intelligence teams to develop an audience analytics platform and recommender systems enabling discoverability and personalization in digital media products. Dr Soudi completed her Ph.D. in Physics at Washington State University. She holds multiple patents and published numerous peer-reviewed papers in high-impact journals. Her current research interests intersect responsible machine learning, sustainability and media.
Student roles:
This role offers a unique opportunity for hands-on research experience. The student will:
Gain a deep understanding of current research challenges in information retrieval, recommender systems, human-computer interaction (HCI), and the socio-technical aspects of the creator economy.
Acquire practical experience with research methodologies, specifically algorithmic auditing, persona development.
Enhance critical thinking skills through engagement with conceptual framework development.
Improve organizational, communication, and teamwork skills within a research environment.
Gain insight into the ethical considerations involved in designing technology with multiple stakeholders.
The undergraduate students will act as a Research Assistant providing vital support to the project. Working under the guidance of senior researchers, the student's responsibilities will include:
Summarizing and synthesizing research articles and reports related to fairness, agency, recommender systems, and the creator economy.
Contributions to Persona Definition: designs unique, automated bots (sock puppets) and assigns them highly specific "viewing" or "browsing" histories.
Assisting with the organization and analysis of audit data.
Assisting in organizing, visualizing, and interpreting preliminary findings from persona development and algorithm audit on media platforms.
Maintaining meticulous records of analyzed content, data sources, and analytical steps.
Participating in regular team meetings, contributing to methodological discussions, and potentially presenting initial findings to the research group.
Potential Deliverables will include:
A documented framework with creator persona based on Canadian media diversity
A cleaned and organized dataset of results from audie of algorithm for each persona
A preliminary report summarizing and quantifying the biases affecting discoverability of content creators in Canada based on audit data , co-developed with the research team.
Skills required:
While prior experience in academic research is not mandatory, we are looking for a motivated student who possesses the following qualities:
Excellent research and critical thinking skills, experience with academic writing is an asset.
Knowledge of web scrapping for behaviour simulation, and managing github repository
Knowledge of recommender systems is a plus
Effective communication skills: The ability to communicate the research findings clearly and effectively both written and orally both technical and non-technical audiences
Time Management: Proper time management skills to meet deadlines and complete the study within the specified timeline.
Strong interest in responsible machine learning, computational social science