parse PDF by keyword GH.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "7da364b0",
   "metadata": {},
   "source": [
    "### This notebook uses a list of keywords from a csv file to parse apart a PDF file. In this example, it takes a PDF file containing sample resumes and parses it apart based on a list (.csv) of the applicants' names. This matching is also improved by using fuzzy matching techniques so that the names in the .csv are used to determine the actual format of the names used in the pdf. \n",
    "\n",
    "##### Note: each individual set of resumes should be saved in it's own folder if planning to pickle results. The reason is that running this code without editing the pickle names can re-write over prior pickles. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "658846ac",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Current working directory is: /Users/swilson/Library/CloudStorage/OneDrive-Personal/python/python notebooks/GitHub/Parse-NLP-ML-main\n"
     ]
    }
   ],
   "source": [
    "#EDIT THIS IF NEW DATA - change directory\n",
    "\n",
    "#import python packages and change directory to the correct folder\n",
    "\n",
    "import pickle\n",
    "import os\n",
    "import pandas\n",
    "import re\n",
    "import pdfminer\n",
    "from pdfminer.high_level import extract_text\n",
    "os.chdir(r'/Users/swilson/Library/CloudStorage/OneDrive-Personal/python/python notebooks/GitHub/Parse-NLP-ML-main')\n",
    "cwd = os.getcwd()\n",
    "print(\"Current working directory is:\", cwd)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "7ae48363",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'RESUMES\\xa0\\n\\nSAMPLES\\n\\nVarious sample resumes to help you see\\nways to improve your resume.\\n\\n\\x0cTABLE OF\\nTABLE OF\\n\\nCONTENTS\\nCONTENTS\\n\\nArchitecture + Planning...........................................................2\\nArchitecture + Planning...........................................................2\\nArchitecture + Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2\\n\\nCultural and Social Transformation...........................4\\nCultural and Social Transformation...........................4\\nCultural and Social Transformation . . . . . . . . . . . . . . . 4\\n\\nEducation............................................................................................5\\nEducation............................................................................................5\\nEducation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5\\n\\nEngineering........................................................................................7\\nEngineering........................................................................................7\\nEngineering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7\\nFine Arts...............................................................................................10\\nFine Arts...............................................................................................10\\nFine Arts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10\\nHealth.....................................................................................................13\\nHealth.....................................................................................................13\\nHealth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13\\nHumanities........................................................................................14\\nHumanities........................................................................................14\\nHumanities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16\\nMines and Earth Sciences...................................................17\\nMines and Earth Sciences...................................................17\\nMines and Earth Sciences . . . . . . . . . . . . . . . . . . . . . . . .19\\nNursing.................................................................................................19\\nNursing.................................................................................................19\\nNursing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21\\nScience.................................................................................................21\\nScience.................................................................................................21\\nScience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .23\\nSocial and Behavioral Sciences....................................23\\nSocial and Behavioral Sciences....................................23\\nSocial and Behavioral Sciences . . . . . . . . . . . . . . . . . . 25\\nSocial Work......................................................................................26\\nSocial Work......................................................................................26\\nSocial Work . . . . . .  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28\\n\\nD I S C L A I M E R\\nD I S C L A I M E R\\n\\nThis booklet is meant to serve as a starting point for students looking for\\nThis booklet is meant to serve as a starting point for students looking for\\nexamples of ways to format their resume. These samples are not meant to\\nexamples of ways to format their resume. These samples are not meant to\\nconfine a student to one design or suggest that all majors have direct\\nconfine a student to one design or suggest that all majors have direct\\ncareer paths. This collection of samples simply serves as a guide. While\\ncareer paths. This collection of samples simply serves as a guide. While\\nresumes are subjective, we have created and adapted these samples to\\nresumes are subjective, we have created and adapted these samples to\\nnationwide best practices.\\nnationwide best practices.\\n\\n\\x0cARCHITECTURE & PLANNING\\n\\nARCHITECHTURE STUDENT\\n123 President’s Circle, Salt Lake City, UT | name@gmail.com | www.myportfolio.com | 000.123.4567\\n\\nARCHITECHTURE STUDENT\\n\\nI\\n\\nN\\nO\\nT\\nA\\nC\\nU\\nD\\nE\\n\\nI\\n\\nE\\nC\\nN\\nE\\nR\\nE\\nP\\nX\\nE\\n\\nI\\n\\nP\\nH\\nS\\nR\\nE\\nB\\nM\\nE\\nM\\n\\nS\\nL\\nL\\nK\\nS\\n\\nI\\n\\nMaster of Architecture | University of Utah | Salt Lake City, UT (May 20xx) \\n\\nBachelor of Science in Architecture | University of  Utah | Austin, TX (May 20xx)\\nMinor in Spanish \\n\\nPublic Interest Design Summer Studio | University of Texas | Austin, TX (Many 20xx-July20xx)\\n\\nThree Cities Studio Study Abroad | Syracuse University | (May 20xx-Aug 20xx)\\nItaly, England, and the Netherlands\\n\\nPublic Interest Design Fellow | Tulane City Center | New Orleans, LA (Jan-Oct 20xx)\\n\\n• Developed the Hollygrove Greenline Master Plan with other design professionals & community stakeholders through schematic \\ndesign and design development phases.\\n• Managed the schedule, budget, material orders, and staff and volunteer labor for installation of five rain gardens on the proper-\\nties of five Hollygrove Neighborhood families, according to the designs of a landscape architect. The project was completed at \\nbudget and on schedule.\\n• Conducted a workshop for Hollygrove residents concerning low impact development education & implementation, including a \\ndemonstration of simple, at-home site surveying and soil percolation testing.  \\n\\nAmeriCorps Capacity Corps Member | Rebuilding Together | Alexandria, VA (Jan-Dec 20xx)\\nCommunity Engagement Coordinator\\nProgram Coordinator (full-time staff) | Rebuilding Together | Alexandria, VA (Jan-Dec 20xx)\\n\\n• Gained experience in skills-based volunteer management, grant writing and fund-raising, and home rehabilitation prioritizing  \\non critical reparis and energy efficiency.\\n• Created a sustainable outreach system that categorized presentations, workshops, and communications conducted by staff \\nmembers, therby expanding the client base and coummunity partnerships.\\n• Participated in collaborative meetings, educational seminars, and conferences focusing on housing issues and related services.  \\n\\nCAD/CNC Technician | OEC Engineering Corp | Chantilly, VA (Mar-Dec 20xx)\\n\\n• Analyzed construction documents to prepare bids for a range of scopes and scales in architectural and engineering fabrication, \\nfrom $100 to multi-million dollar projects.\\n• Collaborated in design and detailing of custom fabrication projects, such as a stone “carpet”, medical equipment, signage and \\nlogos, and furniture.\\n• Programmed code and configuration settings for water jet cutting an array of materials, including glass, metal, plastic, and \\nstone. \\n\\nDesign Consultant | Blinds to Go | Springfield, VA (Sept-Mar 20xx)\\n\\n• Guided customers in selecting the window covering that best represented their desired function, decor, and budget using product \\nknowledge andquality comparison.\\n• Addressed customer service issues and performed repairs on blinds and shades.\\n\\nUSGBC Member (Dec 20xx-Present)\\nSEED Network Member (Mar 20xx-Present)\\nRichardson Memorial Hall Working Group (Fall 20xx)\\nGraduate Admissions Committee (Spr 20xx)\\n\\nD\\nE\\nR\\nC\\nC\\nA\\n\\nLEED BD+C Accredited (expected June 20xx)\\nSEED Accredited (Mar 20xx)\\nLead Safe Renovation Accredited (Jan 20xx)\\n\\nDesign:\\nProject Management (on the job)\\nSchematic Design\\nDesign Development\\nCost Estimation/Budgeting\\n3D & Physical Modeling\\nDrafting & Sketching\\n\\nConstruction & Fabrication:\\nProject Management (on the job)\\nCost Estimation/Budgeting\\nLabor & Resource Management (on the job)\\nCNC Fabrication (3D printer, laser cutter, \\nwater jet cutter, 3 point router)\\nWood & Metal Working\\n\\nSoftware:\\nRevit\\nAutoCAD\\nRhino 5 and VRay\\nAdobe CS6\\nSketchup8\\nESRI ArchMap (GIS)\\n\\nPAGE 2\\nPAGE 2\\n\\n\\x0cARCHITECTURE & PLANNING\\n\\nUrban\\tEcology\\tStudent\\t\\n\\nSalt\\tLake\\tCity,\\tUtah\\t|\\t(555)555-5555\\t|\\turbanecologystudent@gmail.com\\t\\n\\nEDUCATION\\t\\n\\nB.S.\\tUrban\\tEcology,\\tMinor\\tStrategic\\tCommunication\\t\\nUniversity\\tof\\tUtah\\t\\n\\nMajor\\tGPA:\\t3.9\\t\\nDean’s\\tList\\t2014\\t–\\t2018\\t\\n\\nCum.\\tGPA:\\t3.7\\t\\n\\n\\t December\\t2018\\t\\nSalt\\tLake\\tCity,\\tUtah\\t\\n\\nRelevant\\tCoursework\\t\\nEcological\\tPlanning\\tWorkshop\\t\\nGIS\\t&\\tCartography\\t(IP)\\t\\n\\nRELEVANT\\tSKILLS\\t\\n\\nUrban\\tEnvironmental\\tTheory\\t&\\tPractice\\t\\nResearch\\tMethods\\tin\\tUrban\\tEcology\\t\\n\\nLandscape\\tArchitecture\\t(IP)\\t\\nEcology\\tfor\\tPlanners\\t\\n\\nAdobe\\tInDesign\\t\\n\\nAdobe\\tIllustrator\\t\\n\\nAdobe\\tPhotoshop\\t\\n\\nGoogle\\tSketch\\tUp\\t\\n\\nSalesforce\\t\\n\\nRELEVANT\\tPROJECT\\t\\n\\nVisioning\\tDocument\\tCreation\\tfor\\tBryce\\tCity\\tCanyon,\\tUtah\\t\\n\\n•  Collaborated\\twith\\tundergraduate\\tstudents\\tto\\tcreate\\tand\\tdeliver\\ta\\tvisioning\\tdocument\\tfor\\tBryce\\tCanyon\\tCity,\\tUtah.\\t\\n•  Established\\tdevelopment\\tand\\tdesign\\tguidelines\\tfor\\tdesirable\\tgrowth\\toutcomes\\tto\\tincrease\\tfuture\\tproject\\tefficiency.\\t\\n•  Directed\\ta\\tsurvey\\tadministered\\tto\\tresidents\\tand\\tvisitors\\tto\\tanalyze\\tcommunity’s\\tshared\\tvalues\\tto\\tinform\\tvision.\\t\\n\\nPROFESSIONAL\\tEXPERIENCE\\t\\n\\nUndergraduate\\tResearcher\\t\\nUniversity\\tof\\tUtah:\\tOffice\\tof\\tUndergraduate\\tResearch\\t\\n\\nJanuary\\t2018\\t–\\tPresent\\t\\nSalt\\tLake\\tCity,\\tUtah\\t\\n\\n•  Designed\\tand\\tdeveloped\\ta\\tcase\\tstudy\\tof\\tSalt\\tLake\\tCounty\\tbicycle\\tinfrastructure\\tutilizing\\tcensus\\tblock\\tdata.\\t\\n•  Conducted\\tmultiple\\tmode\\tcounts,\\tcoded\\tbuilt\\tenvironment’s\\tcharacteristics,\\tand\\tadministered\\ta\\tquantitative\\t\\n\\nsurvey\\tto\\tselected\\tneighborhood\\tresidents.\\t\\n\\nCommunity\\tRelations\\tIntern\\twith\\tUniversity\\tof\\tUtah’s\\tHinckley\\tInstitute\\tof\\tPolitics\\t\\nSalt\\tLake\\tCounty\\tMayor’s\\tOffice\\t\\n\\nJanuary\\t2018\\t–\\tPresent\\t\\nSalt\\tLake\\tCity,\\tUtah\\t\\n\\n•  Facilitated\\toversight\\tof\\tthe\\tcounty’s\\tboards\\tand\\tcommissions\\tto\\tensure\\tthe\\tMayor’s\\tOffice\\twas\\tup-to-date.\\t\\n•  Augmented\\tcontent\\tfor\\tYouth\\tGovernment\\tprogram\\tand\\torganized\\ta\\tcommunity\\tevents\\tcalendar\\tfor\\tthe\\tMayor’s\\t\\n\\nstaff.\\t\\n\\nSales\\tAssociate\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\tDecember\\t2015-\\tPresen'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#EDIT THIS IF NEW DATA - change pdf name\n",
    "#convert pdf to machine readable format. Takes a few minutes depending on size of pdf\n",
    "raw_text = extract_text('Resume Booklet.pdf')\n",
    "raw_text[:10000]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "8b3babb9",
   "metadata": {},
   "outputs": [],
   "source": [
    "# optional code to pickle the resumes in machine readable format\n",
    "#F = open('raw_text.pkl', 'wb')\n",
    "#pickle.dump(raw_text, F)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "43cf13db",
   "metadata": {},
   "outputs": [],
   "source": [
    "#optional code to pull in the pickled resumes \n",
    "#F = open('raw_text.pkl','rb')\n",
    "#raw_text = pickle.load(F)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "7591b992",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Architecture Student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Urban Ecology Student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Transform Student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Education Student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Education Student 2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Electric Engineering Student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Biomedical Engineering Student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Mining Engineering Student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Graphic Design Student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Art Student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Games Student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Health Promotion and Education Student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Kinesiology Student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Parks Recreation and Tourism Student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>International Studies Student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>Asian Studies Student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>Communication Student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>Mining Engineering Student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>Environmental Science Student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>Nursing Student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>Biology Student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>Chemistry Student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>Psychology Student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>Health Sociology and Policy Student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>Social Work Student</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                       name\n",
       "0                      Architecture Student\n",
       "1                     Urban Ecology Student\n",
       "2                         Transform Student\n",
       "3                         Education Student\n",
       "4                       Education Student 2\n",
       "5              Electric Engineering Student\n",
       "6            Biomedical Engineering Student\n",
       "7                Mining Engineering Student\n",
       "8                    Graphic Design Student\n",
       "9                               Art Student\n",
       "10                            Games Student\n",
       "11   Health Promotion and Education Student\n",
       "12                      Kinesiology Student\n",
       "13     Parks Recreation and Tourism Student\n",
       "14            International Studies Student\n",
       "15                    Asian Studies Student\n",
       "16                    Communication Student\n",
       "17               Mining Engineering Student\n",
       "18            Environmental Science Student\n",
       "19                          Nursing Student\n",
       "20                          Biology Student\n",
       "21                        Chemistry Student\n",
       "22                       Psychology Student\n",
       "23      Health Sociology and Policy Student\n",
       "24                      Social Work Student"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# For this specific example, some lines of code were not needed so are commented out. \n",
    "\n",
    "#EDIT THIS IF NEW DATA - pull in new excel list of names. Excel format required is a single column of names, \n",
    "#last name comma first name. There cannot be a column header.\n",
    "\n",
    "#processing the list of names to prepare for the for loop. The names were in the format last, first middle. This \n",
    "#rearranges the names so it's format first middle last. \n",
    "\n",
    "ea_df= pandas.read_excel (r'Resume Booklet names.xlsx', sheet_name='Sheet1', header=None) #read in excel file, list of names\n",
    "ea_df.rename(columns={ea_df.columns[0]: \"name\" }, inplace = True)\n",
    "ea_df = ea_df['name'].str.split(',',expand=True) #split last name and first and middle name in separate columns\n",
    "ea_df.rename(columns={ea_df.columns[0]: \"last\" }, inplace = True) #rename column names\n",
    "ea_df.rename(columns={ea_df.columns[1]: \"first\" }, inplace = True)#rename column names\n",
    "# ea_df[['first','middle']] = ea_df['first'].str.split().str[::1].apply(pandas.Series).fillna('')#separate middle name from first name\n",
    "#ea_df['middle_initial'] = ea_df['middle'].astype(str).str[0]\n",
    "# ea_df = ea_df.drop(ea_df.columns[2], axis=1) #drop middle name\n",
    "ea_df = ea_df.fillna('')#replace NA values for names without middle name as empty\n",
    "# #ea_df['first_middle_initial'] = ea_df['first'] +' '+ ea_df['middle_initial']\n",
    "# # ea_df = ea_df.drop(columns=['first', 'middle_initial']) #drop excess columns\n",
    "# #ea_df = ea_df.rename(columns={'first_middle_initial': 'first'})\n",
    "ea_df = ea_df[['first', 'last']] # order first name column first \n",
    "ea_df[\"name\"] = ea_df[\"first\"] + ' ' + ea_df[\"last\"] #create a new column with first and last name together, in that order\n",
    "ea_df = ea_df[['name']] #keep only the full name column\n",
    "ea_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "8906b33a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['architecture student',\n",
       " 'urban ecology student',\n",
       " 'transform student',\n",
       " 'education student',\n",
       " 'education student 2',\n",
       " 'electric engineering student',\n",
       " 'biomedical engineering student',\n",
       " 'mining engineering student',\n",
       " 'graphic design student',\n",
       " 'art student',\n",
       " 'games student',\n",
       " 'health promotion and education student',\n",
       " 'kinesiology student',\n",
       " 'parks recreation and tourism student',\n",
       " 'international studies student',\n",
       " 'asian studies student',\n",
       " 'communication student',\n",
       " 'mining engineering student',\n",
       " 'environmental science student',\n",
       " 'nursing student',\n",
       " 'biology student',\n",
       " 'chemistry student',\n",
       " 'psychology student',\n",
       " 'health sociology and policy student',\n",
       " 'social work student']"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#convert pythons df list of names into a list and clean up\n",
    "\n",
    "ea_list = [[i] for i in ea_df['name']] #convert the pandas df into a list\n",
    "flattened = [val for sublist in ea_list for val in sublist] #flatten the list above from a list of list format\n",
    "for i, s in enumerate(flattened):  #remove extra spaces that are added when converting from df to list\n",
    "    flattened[i] = \" \".join(s.split())\n",
    "flattened = [x.lower() for x in flattened] #make all names lowercase\n",
    "flattened = [i.lstrip() for i in flattened] #strip out a space \n",
    "flattened"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "df265f40",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['architecture student',\n",
       " 'urban ecology student',\n",
       " 'transform student',\n",
       " 'education student',\n",
       " 'education student 2',\n",
       " 'electric engineering student',\n",
       " 'biomedical engineering student',\n",
       " 'mining engineering student',\n",
       " 'graphic design student',\n",
       " 'art student',\n",
       " 'games student',\n",
       " 'health promotion and education student',\n",
       " 'kinesiology student',\n",
       " 'parks recreation and tourism student',\n",
       " 'international studies student',\n",
       " 'asian studies student',\n",
       " 'communication student',\n",
       " 'mining engineering student',\n",
       " 'environmental science student',\n",
       " 'nursing student',\n",
       " 'biology student',\n",
       " 'chemistry student',\n",
       " 'psychology student',\n",
       " 'health sociology and policy student',\n",
       " 'social work student']"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#EDIT THIS IF NEW DATA - replace ore remove names if known in advance. \n",
    "\n",
    "#remove and convert names missing from excel list of names\n",
    "\n",
    "#name = [x.replace('childers-conner', 'childers') for x in flattened]\n",
    "#name.remove('christopher hershey')\n",
    "#name = [x.replace('osiruphu-el', 'osiruphu') for x in name]\n",
    "\n",
    "# run this if no names removed\n",
    "name = flattened\n",
    "\n",
    "name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "89aadeee",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'resumes   samples  various sample resumes to help you see ways to improve your resume    table of table of  contents contents  architecture   planning                                                           2 architecture   planning                                                           2 architecture   planning                                                         2  cultural and social transformation                           4 cultural and social transformation                           4 cultural and social transformation                               4  education                                                                                            5 education                                                                                            5 education                                                                                  5  engineering                                                                                        7 engineering                                                                                        7 engineering                                                                               7 fine arts                                                                                               10 fine arts                                                                                               10 fine arts                                                                                    10 health                                                                                                     13 health                                                                                                     13 health                                                                                         13 humanities                                                                                        14 humanities                                                                                        14 humanities                                                                               16 mines and earth sciences                                                   17 mines and earth sciences                                                   17 mines and earth sciences                                                19 nursing                                                                                                 19 nursing                                                                   '"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#proccess resume text to make all words lowercase\n",
    "\n",
    "import re\n",
    "\n",
    "text_lower = re.sub(r\"[^a-zA-Z0-9]\", \" \", raw_text.lower()) #make text lowercase\n",
    "\n",
    "text_lower[:2500]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "7dc46d46",
   "metadata": {},
   "outputs": [],
   "source": [
    "# #DO NOT REMOVE!!! This is the original fuzzy matching code that fuzzy matches the list of names to the names in the raw\n",
    "#text. The names from the raw text will now be used to parse the pdf. Originally, I tried the package fuzzy wuzzy \n",
    "#(also called the fuzz), but I think fuzzy wuzzy needs the raw text to be tokenized otherwise it returns individuals\n",
    "#letters not words. Since I need first, middle, and last name, I need the format to be a string not a token. A token\n",
    "#would separate first, middle, and last names in the raw text with a comma. \n",
    "\n",
    "from fuzzysearch import find_near_matches #takes about 1 hour to run\n",
    "\n",
    "results_search = {} # dictionary to store results \n",
    "\n",
    "#max_l_dist=6 appears to be the max distance. dist =7 won't run, even when left for over an hour (maybe several hours?)\n",
    "for i in name:\n",
    "    results_search[i] = find_near_matches(i, text_lower, max_l_dist=6)\n",
    "\n",
    "results_middle_initial = results_search"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "5c31f575",
   "metadata": {},
   "outputs": [],
   "source": [
    "#Optional - pickle results\n",
    "#Fuzzy search can take up to 1 hour to run, so pickling the result for easy access\n",
    "\n",
    "# F = open('results_middle_initial.pkl', 'wb')\n",
    "# pickle.dump(results_middle_initial, F)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "e191aea5",
   "metadata": {},
   "outputs": [],
   "source": [
    "#Optional - pulling in the results from the fuzzy matching \n",
    "#F = open('results_middle_initial.pkl','rb')\n",
    "#results_middle_initial = pickle.load(F)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "d2f740e3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>original</th>\n",
       "      <th>dist</th>\n",
       "      <th>matched</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>architecture student</td>\n",
       "      <td>6</td>\n",
       "      <td>architecture   plann</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>architecture student</td>\n",
       "      <td>6</td>\n",
       "      <td>architecture   plann</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>architecture student</td>\n",
       "      <td>6</td>\n",
       "      <td>architecture   plann</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>architecture student</td>\n",
       "      <td>6</td>\n",
       "      <td>architecture   plann</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>architecture student</td>\n",
       "      <td>1</td>\n",
       "      <td>architechture student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1396</th>\n",
       "      <td>social work student</td>\n",
       "      <td>6</td>\n",
       "      <td>oint for student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1397</th>\n",
       "      <td>social work student</td>\n",
       "      <td>6</td>\n",
       "      <td>cil on student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1398</th>\n",
       "      <td>social work student</td>\n",
       "      <td>0</td>\n",
       "      <td>social work student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1399</th>\n",
       "      <td>social work student</td>\n",
       "      <td>2</td>\n",
       "      <td>socialworkstudent</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1400</th>\n",
       "      <td>social work student</td>\n",
       "      <td>5</td>\n",
       "      <td>social work  uni</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1401 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                  original dist                 matched\n",
       "0     architecture student    6    architecture   plann\n",
       "1     architecture student    6    architecture   plann\n",
       "2     architecture student    6    architecture   plann\n",
       "3     architecture student    6    architecture   plann\n",
       "4     architecture student    1   architechture student\n",
       "...                    ...  ...                     ...\n",
       "1396   social work student    6        oint for student\n",
       "1397   social work student    6          cil on student\n",
       "1398   social work student    0     social work student\n",
       "1399   social work student    2       socialworkstudent\n",
       "1400   social work student    5        social work  uni\n",
       "\n",
       "[1401 rows x 3 columns]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#processing the fuzzy matched names.\n",
    "\n",
    "fuzz_df = pandas.DataFrame.from_dict(results_middle_initial, orient='index') #convert dictionary to pandas dataframe\n",
    "fuzz_df = fuzz_df.stack().explode() \n",
    "fuzz_df = fuzz_df.to_frame().reset_index()#convert pandas series to dataframe\n",
    "fuzz_df = fuzz_df.rename(columns={ fuzz_df.columns[0]: \"original\", fuzz_df.columns[2]: \"results\" }) #rename columns by position\n",
    "fuzz_df = fuzz_df.drop(fuzz_df.columns[[1]], axis=1) #drop excess column\n",
    "fuzz_df['results'] = fuzz_df['results'].astype(str).agg(lambda x:x.str.strip(\"Match()\")) #removes Match() string from results column\n",
    "fuzz_df[['start','end','dist','matched']] = fuzz_df['results'].str.split(',', expand = True) #split results\n",
    "fuzz_df = fuzz_df.drop(['results','start','end'], axis=1) #drop multiple coulmns by name\n",
    "fuzz_df['dist'] = fuzz_df['dist'].str.replace(\"dist=\",\"\") #remove \"dist=\" from dist column\n",
    "fuzz_df['matched'] = fuzz_df['matched'].str.replace(\"matched='\",\"\")#remove \"matched=''\" from matched column\n",
    "fuzz_df['matched'] = fuzz_df['matched'].str.replace(\"'\",\"\")#remove ' from end of name in matched column\n",
    "fuzz_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "6275c864",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>original</th>\n",
       "      <th>dist</th>\n",
       "      <th>matched</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>architecture student</td>\n",
       "      <td>1</td>\n",
       "      <td>architechture student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>urban ecology student</td>\n",
       "      <td>0</td>\n",
       "      <td>urban ecology student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>transform student</td>\n",
       "      <td>0</td>\n",
       "      <td>transform student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>education student</td>\n",
       "      <td>0</td>\n",
       "      <td>education student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>education student 2</td>\n",
       "      <td>0</td>\n",
       "      <td>education student 2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>electric engineering student</td>\n",
       "      <td>2</td>\n",
       "      <td>electrical engineering student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>biomedical engineering student</td>\n",
       "      <td>0</td>\n",
       "      <td>biomedical engineering student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>mining engineering student</td>\n",
       "      <td>0</td>\n",
       "      <td>mining engineering student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>graphic design student</td>\n",
       "      <td>0</td>\n",
       "      <td>graphic design student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>art student</td>\n",
       "      <td>0</td>\n",
       "      <td>art student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>games student</td>\n",
       "      <td>2</td>\n",
       "      <td>gamestudent</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>health promotion and education student</td>\n",
       "      <td>0</td>\n",
       "      <td>health promotion and education student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>kinesiology student</td>\n",
       "      <td>0</td>\n",
       "      <td>kinesiology student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>parks recreation and tourism student</td>\n",
       "      <td>6</td>\n",
       "      <td>parks  recreation and tourism   uni</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>international studies student</td>\n",
       "      <td>0</td>\n",
       "      <td>international studies student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>asian studies student</td>\n",
       "      <td>0</td>\n",
       "      <td>asian studies student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>communication student</td>\n",
       "      <td>0</td>\n",
       "      <td>communication student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>environmental science student</td>\n",
       "      <td>3</td>\n",
       "      <td>environmental geoscience student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>nursing student</td>\n",
       "      <td>0</td>\n",
       "      <td>nursing student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>biology student</td>\n",
       "      <td>0</td>\n",
       "      <td>biology student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>chemistry student</td>\n",
       "      <td>0</td>\n",
       "      <td>chemistry student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>psychology student</td>\n",
       "      <td>0</td>\n",
       "      <td>psychology student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>health sociology and policy student</td>\n",
       "      <td>6</td>\n",
       "      <td>health  society  and policy student</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>social work student</td>\n",
       "      <td>0</td>\n",
       "      <td>social work student</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                  original  dist  \\\n",
       "0                     architecture student     1   \n",
       "1                    urban ecology student     0   \n",
       "2                        transform student     0   \n",
       "3                        education student     0   \n",
       "4                      education student 2     0   \n",
       "5             electric engineering student     2   \n",
       "6           biomedical engineering student     0   \n",
       "7               mining engineering student     0   \n",
       "8                   graphic design student     0   \n",
       "9                              art student     0   \n",
       "10                           games student     2   \n",
       "11  health promotion and education student     0   \n",
       "12                     kinesiology student     0   \n",
       "13    parks recreation and tourism student     6   \n",
       "14           international studies student     0   \n",
       "15                   asian studies student     0   \n",
       "16                   communication student     0   \n",
       "17           environmental science student     3   \n",
       "18                         nursing student     0   \n",
       "19                         biology student     0   \n",
       "20                       chemistry student     0   \n",
       "21                      psychology student     0   \n",
       "22     health sociology and policy student     6   \n",
       "23                     social work student     0   \n",
       "\n",
       "                                    matched  \n",
       "0                     architechture student  \n",
       "1                     urban ecology student  \n",
       "2                         transform student  \n",
       "3                         education student  \n",
       "4                       education student 2  \n",
       "5            electrical engineering student  \n",
       "6            biomedical engineering student  \n",
       "7                mining engineering student  \n",
       "8                    graphic design student  \n",
       "9                               art student  \n",
       "10                              gamestudent  \n",
       "11   health promotion and education student  \n",
       "12                      kinesiology student  \n",
       "13      parks  recreation and tourism   uni  \n",
       "14            international studies student  \n",
       "15                    asian studies student  \n",
       "16                    communication student  \n",
       "17         environmental geoscience student  \n",
       "18                          nursing student  \n",
       "19                          biology student  \n",
       "20                        chemistry student  \n",
       "21                       psychology student  \n",
       "22      health  society  and policy student  \n",
       "23                      social work student  "
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Keep only the matches with minimum levenshtein distance \n",
    "\n",
    "fuzz_df['dist'] = pandas.to_numeric(fuzz_df['dist']) #convert dist column from obj to numeric\n",
    "min_dist = fuzz_df.loc[fuzz_df.groupby('original').dist.idxmin()] #group by each original name and keep the minimum dist\n",
    "min_dist.sort_index(inplace=True) #sort by the index\n",
    "min_dist = min_dist.reset_index(drop=True) #reset the index \n",
    "min_dist"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "780fe3dc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.8333333333333334"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "min_dist[\"dist\"].mean() #with middle names, mean is 1.58. Without middle names is 1.265"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "f6c78b9e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "20"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "min_dist[\"dist\"].sum()  #with middle names, sum is 76. Without middle names is 62"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "65179fd9",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'architechture student': 'urban ecology student',\n",
       " 'urban ecology student': 'transform student',\n",
       " 'transform student': 'education student',\n",
       " 'education student': 'education student 2',\n",
       " 'education student 2': 'electrical engineering student',\n",
       " 'electrical engineering student': 'biomedical engineering student',\n",
       " 'biomedical engineering student': 'mining engineering student',\n",
       " 'mining engineering student': 'graphic design student',\n",
       " 'graphic design student': 'art student',\n",
       " 'art student': 'gamestudent',\n",
       " 'gamestudent': 'health promotion and education student',\n",
       " 'health promotion and education student': 'kinesiology student',\n",
       " 'kinesiology student': 'parks  recreation and tourism   uni',\n",
       " 'parks  recreation and tourism   uni': 'international studies student',\n",
       " 'international studies student': 'asian studies student',\n",
       " 'asian studies student': 'communication student',\n",
       " 'communication student': 'environmental geoscience student',\n",
       " 'environmental geoscience student': 'nursing student',\n",
       " 'nursing student': 'biology student',\n",
       " 'biology student': 'chemistry student',\n",
       " 'chemistry student': 'psychology student',\n",
       " 'psychology student': 'health  society  and policy student',\n",
       " 'health  society  and policy student': 'social work student'}"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#repeating conversion from dataframe to dictionary\n",
    "min_dist_match = min_dist.drop(['original','dist'], axis=1) # keep only the matched column of names\n",
    "match_list = [[i] for i in min_dist_match['matched']] #convert dataframe to list\n",
    "match_flattened = [val for sublist in match_list for val in sublist] #flatten list\n",
    "match_flattened = [i.lstrip() for i in match_flattened] #remove spaces\n",
    "match_pairs = list(zip(match_flattened, match_flattened[1:]))#pair names\n",
    "match_pairs = dict(match_pairs) #converting list to dictionary\n",
    "\n",
    "match_pairs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "b3f0c6c0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "OrderedDict([('architechture student', 'urban ecology student'),\n",
       "             ('urban ecology student', 'transform student'),\n",
       "             ('transform student', 'education student'),\n",
       "             ('education student', 'education student 2'),\n",
       "             ('education student 2', 'electrical engineering student'),\n",
       "             ('electrical engineering student',\n",
       "              'biomedical engineering student'),\n",
       "             ('biomedical engineering student', 'mining engineering student'),\n",
       "             ('mining engineering student', 'graphic design student'),\n",
       "             ('graphic design student', 'art student'),\n",
       "             ('art student', 'gamestudent'),\n",
       "             ('gamestudent', 'health promotion and education student'),\n",
       "             ('health promotion and education student', 'kinesiology student'),\n",
       "             ('kinesiology student', 'parks  recreation and tourism   uni'),\n",
       "             ('parks  recreation and tourism   uni',\n",
       "              'international studies student'),\n",
       "             ('international studies student', 'asian studies student'),\n",
       "             ('asian studies student', 'communication student'),\n",
       "             ('communication student', 'environmental geoscience student'),\n",
       "             ('environmental geoscience student', 'nursing student'),\n",
       "             ('nursing student', 'biology student'),\n",
       "             ('biology student', 'chemistry student'),\n",
       "             ('chemistry student', 'psychology student'),\n",
       "             ('psychology student', 'health  society  and policy student'),\n",
       "             ('health  society  and policy student', 'social work student'),\n",
       "             ('social work student', 'architechture student')])"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#convert from dictionary to Ordered Dictionary in order to capture the last name in the list\n",
    "\n",
    "from collections import OrderedDict\n",
    "\n",
    "order_match_pairs = OrderedDict(match_pairs)#creates ordered dictionary so you can access the last value\n",
    "order_match_pairs[order_match_pairs[next(reversed(order_match_pairs))]] = next(iter(order_match_pairs)) #assign the last value of dictionary as a key and the first key as a value to capture the last name in the list\n",
    "\n",
    "order_match_pairs\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "6731913c",
   "metadata": {},
   "outputs": [],
   "source": [
    "#loop through raw text, parse names into a dictionary\n",
    "\n",
    "parsed_middle_initial = {} #creates a dictionary that the for loop will add values to\n",
    "\n",
    "for key, value in order_match_pairs.items():\n",
    "    if key == next(reversed(order_match_pairs)): #capture the last name until the end of the raw text\n",
    "        parsed_middle_initial[format(key)] = text_lower[text_lower.index(key):] \n",
    "    else: \n",
    "        parsed_middle_initial[format(key)] = text_lower[text_lower.index(key):text_lower.index(value)+1] #all other names"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "3f340f02",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'architechture student 123 president s circle  salt lake city  ut   name gmail com   www myportfolio com   000 123 4567  architechture student  i  n o t a c u d e  i  e c n e r e p x e  i  p h s r e b m e m  s l l k s  i  master of architecture   university of utah   salt lake city  ut  may 20xx    bachelor of science in architecture   university of  utah   austin  tx  may 20xx  minor in spanish   public interest design summer studio   university of texas   austin  tx  many 20xx july20xx   three cities studio study abroad   syracuse university    may 20xx aug 20xx  italy  england  and the netherlands  public interest design fellow   tulane city center   new orleans  la  jan oct 20xx     developed the hollygrove greenline master plan with other design professionals   community stakeholders through schematic  design and design development phases    managed the schedule  budget  material orders  and staff and volunteer labor for installation of five rain gardens on the proper  ties of five hollygrove neighborhood families  according to the designs of a landscape architect  the project was completed at  budget and on schedule    conducted a workshop for hollygrove residents concerning low impact development education   implementation  including a  demonstration of simple  at home site surveying and soil percolation testing     americorps capacity corps member   rebuilding together   alexandria  va  jan dec 20xx  community engagement coordinator program coordinator  full time staff    rebuilding together   alexandria  va  jan dec 20xx     gained experience in skills based volunteer management  grant writing and fund raising  and home rehabilitation prioritizing   on critical reparis and energy efficiency    created a sustainable outreach system that categorized presentations  workshops  and communications conducted by staff  members  therby expanding the client base and coummunity partnerships    participated in collaborative meetings  educational seminars  and conferences focusing on housing issues and related services     cad cnc technician   oec engineering corp   chantilly  va  mar dec 20xx     analyzed construction documents to prepare bids for a range of scopes and scales in architectural and engineering fabrication   from  100 to multi million dollar projects    collaborated in design and detailing of custom fabrication projects  such as a stone  carpet   medical equipment  signage and  logos  and furniture    programmed code and configuration settings for water jet cutting an array of materials  including glass  metal  plastic  and  stone    design consultant   blinds to go   springfield  va  sept mar 20xx     guided customers in selecting the window covering that best represented their desired function  decor  and budget using product  knowledge andquality comparison    addressed customer service issues and performed repairs on blinds and shades   usgbc member  dec 20xx present  seed network member  mar 20xx present  richardson memorial hall working group  fall 20xx  graduate admissions committee  spr 20xx   d e r c c a  leed bd c accredited  expected june 20xx  seed accredited  mar 20xx  lead safe renovation accredited  jan 20xx   design  project management  on the job  schematic design design development cost estimation budgeting 3d   physical modeling drafting   sketching  construction   fabrication  project management  on the job  cost estimation budgeting labor   resource management  on the job  cnc fabrication  3d printer  laser cutter   water jet cutter  3 point router  wood   metal working  software  revit autocad rhino 5 and vray adobe cs6 sketchup8 esri archmap  gis   page 2 page 2   architecture   planning  u'"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#displaying only the first value, the resume associated with the key \"architure student\"\n",
    "parsed_middle_initial['architechture student']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "796fe79a",
   "metadata": {},
   "outputs": [],
   "source": [
    "#optional code to search for specific names\n",
    "#parsed_middle_initial['laura m  wilson']#[-1000:-1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "9803288a",
   "metadata": {},
   "outputs": [],
   "source": [
    "#pickle results to use in resume analysis\n",
    "F = open('parsed_middle_initial.pkl', 'wb')\n",
    "pickle.dump(parsed_middle_initial, F)\n",
    "F.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "a9e9d983",
   "metadata": {},
   "outputs": [],
   "source": [
    "#pickle min_dist to use in mapping back in resume analysis\n",
    "min_dist.to_pickle(\"min_dist_middle_initial.pkl\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}