{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "<div>\n",
    "    <h1><center>CS105 Mini-Project</center></h1>\n",
    "    <h2><center>Does who a student is living with effect if and how they work jobs?</center></h2>\n",
    "</div>\n",
    "\n",
    "By:\n",
    "- Anshul Gupta <agupt109@ucr.edu>\n",
    "- Ali Naqvi <anaqv007@ucr.edu>\n",
    "- Alex Zhang <azhan061@ucr.edu>\n",
    "- Nathan Lee <nlee097@ucr.edu>"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "2c8a0ad98e8c4a8c"
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Pre-Data Questions & Analysis\n",
    "\n",
    "## What data do we have?\n",
    "\n",
    "We have data regarding the living situations of students across multiple CS classes. We asked them who they live with, how many people they live with, where they live, whether they work, how much they work, etc. There are a mix of categorical and quantitative datapoints that we will analyze to answer what we want to know:\n",
    "\n",
    "## What do we want to know?\n",
    "\n",
    "We want to know how various aspects of a student’s home environment go on to affect their employment and school performance. Specifically who they live with and the affects of that.\n",
    "\n",
    "## Hypothesis & Predictions\n",
    "\n",
    "- Hypothesis 1: There will be a correlation between whether people live with family, friends, or neither and whether or not they work.\n",
    "- Hypothesis 2: Students who live on-campus are more likely to have roommates of the same major.\n",
    "- Hypothesis 3: People who live with more people will have a higher GPA on average.\n"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "c080b97dd7add50a"
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Data Loading & Preprocessing"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "e892e92994efdbca"
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import seaborn as sns\n",
    "import matplotlib\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Load dataframe from data.csv\n",
    "df = pd.read_csv(\"data.csv\")\n",
    "\n",
    "# Select relevant columns\n",
    "df = df.iloc[:, [0, 2, 3, 7, 8, 9, 34, 55, 58, 59, 60, 61, 62, 26, 66]]\n",
    "df"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "8df0ee871a2c9a29"
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Preprocessing"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "d5645d5671f8f42c"
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "# Fixes empty values\n",
    "df['Do you currently work?'] = df['Do you currently work?'].fillna('No')\n",
    "\n",
    "# Replaces custom text answers with appropriate values\n",
    "df['How many people live in your household?'] = (df['How many people live in your household?']\n",
    "                                                 .fillna(0)\n",
    "                                                 .replace('4 in total', '4')\n",
    "                                                 .replace('4 (Including me)', '4')\n",
    "                                                 .replace('at school 4 including me ', '4')\n",
    "                                                 .replace('3 excluding me', '4')\n",
    "                                                 .replace('5 including me', '5')\n",
    "                                                 .replace('North District 4 bed 2 bath', '4')\n",
    "                                                 .replace('3 (room), 8 (hall), ~70 (building)', '3')\n",
    "                                                 .astype(int))\n",
    "df['Who do you live with?'] = df['Who do you live with?'].replace('Family, Friends', 'Both').replace(\n",
    "    'Family, Friends, Both', 'Both')\n",
    "df['Do you currently live in a house, apartment, or dorm?'] = (\n",
    "    df['Do you currently live in a house, apartment, or dorm?']\n",
    "    .replace('house (renting)', 'House'))\n",
    "\n",
    "df.loc[df['What was your GPA your very first quarter at UCR?'].str.contains(\n",
    "    \"I am not sure|idk|I don't know|This is my first quarter|i don't rem|not sure|I never checked. |I dont know\") == True, 'What was your GPA your very first quarter at UCR?'] = np.nan\n",
    "df['What was your GPA your very first quarter at UCR?'] = (\n",
    "    df['What was your GPA your very first quarter at UCR?']\n",
    "    .replace('Idk, I think 3.2 or something along those lines', '3.2')\n",
    "    .replace('2.8?', '2.8')\n",
    "    .replace('3 point something', '3.0')\n",
    "    .replace('3.67 I think', '3.67')\n",
    "    .replace('3.0?', '3.0')\n",
    "    .replace('about 3.0', '3.0')\n",
    "    .astype(np.float64))\n",
    "\n",
    "\n",
    "df.loc[df['How many internship/job applications have you sent out so far?'].str.contains(\n",
    "    \"A lot|idk|I don't know|More than enough|Not enough|Many|not sure|I dont know\") == True, 'How many internship/job applications have you sent out so far?'] = np.nan\n",
    "df['How many internship/job applications have you sent out so far?'] = (\n",
    "    df['How many internship/job applications have you sent out so far?']\n",
    "    .fillna(0)\n",
    "    .replace('100s', '100')\n",
    "    .replace(\n",
    "        'I haven’t sent any internships I think I need to take more courses that are CS related then apply to internships to have a better chance to be accepted ',\n",
    "        '0')\n",
    "    .replace('none :(', '0')\n",
    "    .replace('15+', '15')\n",
    "    .replace('none for now', '0')\n",
    "    .replace('20+', '20')\n",
    "    .replace('Above 50 for this summer but overall over the last 4 years over a thousand', '1000')\n",
    "    .replace('5-10', '7')\n",
    "    .replace('200+', '200')\n",
    "    .replace('50+', '50')\n",
    "    .replace('none', '0')\n",
    "    .replace('25+', '25')\n",
    "    .replace('~20', '20')\n",
    "    .replace('100+', '100')\n",
    "    .replace('50-80 this year', '65')\n",
    "    .replace('300+', '300')\n",
    "    .replace('30-40', '35')\n",
    "    .replace('~60', '60')\n",
    "    .replace('between 50-100', '75')\n",
    "    .replace('Less than 10 :( Too many things to do', '10')\n",
    "    .replace('150-200', '175')\n",
    "    .replace('>100', '100')\n",
    "    .replace('~50', '50')\n",
    "    .replace('Over 20', '20')\n",
    "    .replace('10-20', '15')\n",
    "    .astype(int))\n",
    "# Normalizes non-applicable answers\n",
    "df.loc[df['Do you currently work?'] == 'No', 'How many hours do you work per week on average?'] = 0\n",
    "df.loc[df['Do you currently work?'] == 'No', 'Do you work in a department related to your major?'] = np.nan\n",
    "\n",
    "df"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "f802771cabd577a6"
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "# Working DataFrame\n",
    "w_df = df[df['Do you currently work?'] == 'Yes']\n",
    "# Not working DataFrame\n",
    "nw_df = df[df['Do you currently work?'] == 'No']\n",
    "w_df"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "383d33cce226b231"
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "nw_df"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "acd7c65ace4f8c6e"
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Analysis"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "7257425fd2861d86"
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "# Count the number of people who work and don't work\n",
    "work_counts = df['Do you currently work?'].value_counts()\n",
    "\n",
    "# Plotting a pie chart\n",
    "plt.figure(figsize=(8, 8))\n",
    "plt.pie(work_counts, labels=work_counts.index, autopct='%1.1f%%', startangle=90, colors=['lightblue', 'lightcoral'])\n",
    "plt.title('Distribution of People Who Work and Don\\'t Work')\n",
    "plt.show()"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "18123e916da5be69"
  },
  {
   "cell_type": "markdown",
   "source": [
    "The majority of student respondents (70.4%) do **not** work while attending school."
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "9ff9915117dd626b"
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "df_2dhist = pd.crosstab(df.loc[:, 'Do you currently work?'],\n",
    "                        df.loc[:, 'Do you currently live in a house, apartment, or dorm?'],\n",
    "                        normalize='index')\n",
    "\n",
    "# Plot heatmap\n",
    "plt.subplots(figsize=(8, 8))\n",
    "sns.heatmap(df_2dhist, cmap=\"Blues\")\n",
    "plt.xlabel('Do you currently live in a house, apartment, or dorm?')\n",
    "_ = plt.ylabel('Do you currently work?')\n",
    "df_2dhist"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "b743036599a532b7"
  },
  {
   "cell_type": "markdown",
   "source": [
    "For both working & non-working participants, the proportion who live in an apartment are equivalent (50%).\n",
    "\n",
    "However, 13% of non-working participants live in a dorm while only 6% of working participants live in a dorm.\n",
    "This 7% drop is matched in participants who live a house, with 44% of working participants living in a house compared to 36% of non-working participants.\n",
    "\n",
    "This indicates that working participants tend to live off-campus and in living situations that have a higher cost of living."
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "7858aea737ed30d4"
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "df.groupby('Do you currently live in a house, apartment, or dorm?').size().plot(kind='barh',\n",
    "                                                                                color=sns.palettes.mpl_palette(\n",
    "                                                                                    'Dark2'))\n",
    "plt.gca().spines[['top', 'right', ]].set_visible(False)"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "47ff769a519d9481"
  },
  {
   "cell_type": "markdown",
   "source": [
    "Most participants live in either an Apartment or a House. This would indicate that most students either live off-campus or on-campus apartments."
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "20cc959e313e6e3"
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "dataTable1 = pd.pivot_table(data=df, values='What was your GPA your very first quarter at UCR?',\n",
    "                            index='How many hours do you work per week on average?', aggfunc='mean')\n",
    "_ = dataTable1.plot(kind='bar')\n",
    "print(\"Total Average GPA: \", df['What was your GPA your very first quarter at UCR?'].mean())"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "2906701e016c4ae9"
  },
  {
   "cell_type": "markdown",
   "source": [
    "The average GPA seems to be independent in respect to working hours per week.\n",
    "Most students who work less than 20 hours have an equivalent average GPA to the total average GPA of all participants (3.65).\n",
    "\n",
    "There is a small drop in GPA associated with students who work more than 20 hours (3.5 GPA), which may mean some of those students may struggle maintaining balance between work and school.  \n",
    "\n",
    "This would indicate that most students seem to be able to balance work with school. However, it would also indicate that\n",
    "students who work full-time jobs may struggle slightly in school."
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "ef233a2971697aab"
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "stkbar_df = df.dropna()\n",
    "_ = pd.crosstab(\n",
    "    df['Do you have family members who have careers related to your career aspirations?'],\n",
    "    df['What are your career plans right after graduation?'],\n",
    "    normalize='index'\n",
    ").plot(kind=\"bar\", stacked=True, rot=0, figsize=(30, 15))"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "385bbc6ce3896e4b"
  },
  {
   "cell_type": "markdown",
   "source": [
    "Most students across all groups are looking to \"get into the job industry\".\n",
    "Across all groups, the proportions of \"Attending Grad School\" and \"Get into the job industry\" are similar, except for students who have extended family in their career.\n",
    "They are more unsure about their future compared to other groups.\n",
    "Also, students with no family in their field are more diversified in their career plans."
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "d0583ebd2738069e"
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "fig, axes = plt.subplots(nrows=1, ncols=2, figsize=(10, 5))\n",
    "\n",
    "_ = pd.crosstab(\n",
    "    df['What is your current class standing?'],\n",
    "    df['Do you currently work?'],\n",
    ").plot(kind='bar', stacked=True, ax=axes[0])\n",
    "_ = pd.crosstab(\n",
    "    df['What is your current class standing?'],\n",
    "    df['Do you currently work?'],\n",
    "    normalize='index'\n",
    ").plot(kind='bar', stacked=True, ax=axes[1])"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "39456894b80d8dfc"
  },
  {
   "cell_type": "markdown",
   "source": [
    "The class standing most likely to work are graduate students, where 100% of participants work. \n",
    "The freshman class is the least likely to work, where 92% of participants do not work.\n",
    "\n",
    "For Sophomore, Junior, and Senior participants, all 3 groups have similar proportions working with 30% of participants working.  "
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "4a2048ec688f7883"
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "fig, axes = plt.subplots(nrows=1, ncols=2, figsize=(10, 5))\n",
    "\n",
    "_ = pd.crosstab(\n",
    "    w_df['What is your current class standing?'],\n",
    "    w_df['Do you work in a department related to your major?'],\n",
    ").plot(kind='bar', stacked=True, ax=axes[0])\n",
    "\n",
    "_ = pd.crosstab(\n",
    "    w_df['What is your current class standing?'],\n",
    "    w_df['Do you work in a department related to your major?'],\n",
    "    normalize='index',\n",
    ").plot(kind='bar', stacked=True, ax=axes[1])"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "589b08fece5d444f"
  },
  {
   "cell_type": "markdown",
   "source": [
    "Of the students who responded \"yes\" to currently working, the above graphs show the proportions of participants who work in a department related to their major.\n",
    "Most students do not work in a department related to their major, indicating that they are working for money rather than job experience. \n",
    "his holds true for all groups except for Graduate students, who all work in a department of their major.\n",
    "\n",
    "Interestingly, Juniors have a higher rate of working in their major, perhaps indicating internships or students seeking to gain major-related work experience."
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "940f0c4b255ef2c3"
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "print(\"mean internships: \", df[\"How many internship/job applications have you sent out so far?\"].mean())\n",
    "print(\"median internships: \", df[\"How many internship/job applications have you sent out so far?\"].median())\n",
    "_ = sns.violinplot(x=df[\"How many internship/job applications have you sent out so far?\"])"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "4aa4e8ea4584e34"
  },
  {
   "cell_type": "markdown",
   "source": [
    "From the violin plot, we can see that few students send out many job applications. Some students have sent out many (up to 1000) applications while most send about 2.\n",
    "The distribution is skewed right with many outliers who have sent out significantly more applications than most students."
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "1dade2a82270e3ce"
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Hypotheses"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "51a57482103563bd"
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Hypothesis 1: There will be a correlation between whether people live with family, friends, or neither and whether or not they work\n",
    "\n",
    "Null Hypothesis: There is no relationship between people who live with family, friends, or neither and whether or not they work.\n",
    "\n",
    "Significance value: 0.1\n",
    "Degrees of freedom: 3"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "16949928f30a3883"
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "hyp3_major_table = pd.crosstab(df.iloc[:, 3], df.iloc[:, 8], margins=True, margins_name='Total')\n",
    "hyp3_major_table"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "160b9af10bfad29e"
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "num_rows, num_cols = hyp3_major_table.shape\n",
    "# Initialize expected frequencies\n",
    "expected_frequencies = []\n",
    "chi_squared = 0\n",
    "for i in range(num_rows - 1):\n",
    "    row_totals = hyp3_major_table.iloc[i, -1]\n",
    "    for j in range(num_cols - 1):\n",
    "        col_totals = hyp3_major_table.iloc[-1, j]\n",
    "        expected_frequency = (row_totals * col_totals) / hyp3_major_table.iloc[-1, -1]\n",
    "        expected_frequencies.append(expected_frequency)\n",
    "        chi_squared += ((hyp3_major_table.iloc[i, j] - expected_frequency) ** 2) / expected_frequency\n",
    "\n",
    "print(\"Chi-squared value:\", chi_squared)"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "9f6fd2609903cd7e"
  },
  {
   "cell_type": "markdown",
   "source": [
    "With a significance value of 0.1 and 3 degrees of freedom, chi-squared must be greater than 6.25.\n",
    "Since chi-squared of `4.61 < 6.25`, we accept the null hypothesis:\n",
    "\n",
    "There is no relationship between people who live with family, friends, or neither and whether or not they work."
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "6a38c830207ca843"
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Hypothesis 2: Students who live on-campus are more likely to have roommates of the same major.\n",
    "\n",
    "Null Hypothesis: There is no relationship between students who live on-campus and students who have roommates of the same major.\n",
    "\n",
    "Significance value: 0.1\n",
    "Degrees of Freedom: 2"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "72885ea83c45f499"
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "roommates_major_table = pd.crosstab(df.iloc[:, 4], df.iloc[:, 11], margins=True, margins_name='Total')\n",
    "roommates_major_table"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "7cb5d9346986307b"
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "num_rows, num_cols = roommates_major_table.shape\n",
    "# Initialize expected frequencies\n",
    "expected_frequencies = []\n",
    "chi_squared = 0\n",
    "for i in range(num_rows - 1):\n",
    "    row_totals = roommates_major_table.iloc[i, -1]\n",
    "    for j in range(num_cols - 1):\n",
    "        col_totals = roommates_major_table.iloc[-1, j]\n",
    "        expected_frequency = (row_totals * col_totals) / roommates_major_table.iloc[-1, -1]\n",
    "        expected_frequencies.append(expected_frequency)\n",
    "        chi_squared += ((roommates_major_table.iloc[i, j] - expected_frequency) ** 2) / expected_frequency\n",
    "\n",
    "print(\"Chi-squared value:\", chi_squared)"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "de4515b7c575d6a6"
  },
  {
   "cell_type": "markdown",
   "source": [
    "With a significance value of 0.1 and 2 degrees of freedom, chi-squared must be greater than 4.61.\n",
    "Since chi-squared of `4.18 < 4.61`, we accept the null hypothesis:\n",
    "\n",
    "There is no relationship between students who live on-campus and students who have roommates of the same major."
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "48719955c6a9244b"
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Hypothesis 3: People who live with more people will have a higher GPA on average."
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "c20691baf6f8cd9f"
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "hyp3_major_table = pd.crosstab(df.iloc[:, 5], df.iloc[:, 6], margins=True, margins_name='Total')\n",
    "average_household_size = df.iloc[:, 5].mean(skipna=True)\n",
    "average_gpa = df.iloc[:, 6].mean(skipna=True)\n",
    "\n",
    "print(\"Average Household Size:\", average_household_size)\n",
    "print(\"Average GPA:\", average_gpa)\n",
    "numerator = 0\n",
    "denom_x = 0\n",
    "denom_y = 0\n",
    "for i in range(260):\n",
    "    x_i = df.iloc[i, 5]\n",
    "    y_i = df.iloc[i, 6]\n",
    "    if not pd.isna(x_i) and not pd.isna(y_i):  # Check for NaN values\n",
    "        numerator += (x_i - average_household_size) * (y_i - average_gpa)\n",
    "        denom_x += (x_i - average_household_size) ** 2\n",
    "        denom_y += (y_i - average_gpa) ** 2\n",
    "\n",
    "# Calculate Pearson correlation coefficient\n",
    "pearson_coefficient = (numerator / ((denom_x * denom_y) ** 0.5))\n",
    "print(\"Pearson Correlation Coefficient:\", pearson_coefficient)"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "56cca411aaae6ab4"
  },
  {
   "cell_type": "markdown",
   "source": [
    "With a Pearson Correlation Coefficient of -0.2, there is a slight negative correlation between household size and average GPA.\n",
    "Students who live alone or with fewer people perform slightly better than those with more roommates.\n",
    "\n",
    "This goes against our hypothesis that people living with more people will have a higher GPA."
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "d991c7dbec357f45"
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Conclusion\n",
    "\n",
    "In wanting to figure out in general how various aspects of a student’s home environment go on to affect their employment and school performance, we performed 3 different tests: Comparing what kind of people participants lived with and whether or not they work, students who work on campus and their roommates majors, and the number of people participants lived with and the participant’s GPA. However, using chi-squared tests and pearson correlation, we discovered that none of our hypotheses had any correlation with it. But from here we can delve deeper into our hypothesis, for example why did working not affect student’s GPA’s in the first quarter? What are other potential factors as to why that was the case? In addition, Freshmen were found to not work at major related jobs, whereas juniors were the most likely to work at a major related job.\n",
    "\n",
    "Even though none of our hypotheses were proven to be true, we still learned a lot about the data given to us and we can use it to further more questions and assumptions later down the line.\n"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "a9449d76a1dd6523"
  }
 ],
 "metadata": {
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}