Rupal's Updates
Week 1 (February 05 - 11, 2018)
Getting Started
Up to this week, along with the rest of my team, I have mainly been organizing and setting up how the rest of the semester will play out. We met last Saturday (2/03/2018) for a few hours in order to plan out our schedule for the remaining weeks of the semester, detailing what we hope to accomplish and how long each step to the end goal will take. In addition, we collaborated in writing out the requirements, seen here. Since we had a new team member join us, I helped catch her up to speed as to what we discussed with the customers when we met them for the first time, as well as edited the webpage to include her information. Additionally, I have been attempting to get through a TensorFlow tutorial that our customers recommended we test out before delving any deeper into the project. It has proven to be difficult on Windows 10, so I am still in the process of modifying some of the steps given to accommodate my machine.
Week 2 (February 12 - 18, 2018)
Baby Steps
This week, I was finally able to complete the TensorFlow tutorial mentioned in last week's update. Thanks to Donald's suggestion, I used the Google Cloud Shell to finish working through the steps of the tutorial. This allowed us to become more fimiliar with TensorFlow and TensorBoard, as well as get an idea of how to train and test on annotated images. The rest of the week mainly consisted of data collection. We focused on collecting images of four cards: the ace of diamonds, the king of hearts, the queen of hearts, and the jack of hearts. Each of us took 60 pictures of each card in different angles and positions on varying backgrounds. This is in order to create a very diverse training set, allowing for image recognition to be more inclusive. After taking the various images, I labeled each one using an LabelImg, a program created solely to annotate images. A main portion of this project relies on a wide and various data set to train on - to accomplish this, our plan is to repeat this data collection and annotation process during the upcoming weeks for more cards.
Week 3 (February 19 - 25, 2018)
Growing Pains
The whole process of this project is very new to all of us, but we are definitely learning! We quickly realized that our images from last week were way too large for our training model. We decided to re-size our images to a width of 320 pixels, preparing for if we decide to create a mobile application. Because we re-size all of our images, this meant we also had to re-annotate all of our images. Another problem we ran into was the inconsistency amongst all of our labelling formats. To avoid this problem, we communicated to annotate our images by utilizing a modified version of "camel case", in which the first word's first letter is also capitalized (ex: AceOfDiamonds). We focused the rest of our time in preparing for the Midterm Presentation. My team member Adam Walls put together a draft of a PowerPoint presentation earlier this week and I modified and edited it, creating a second version. Adam's version focused on content (he did a wonderful job with this!), while mine was more focused on communicating the information in a concise manner that anyone without prior knowledge of the subject would be able to understand. We planned to meet today to practice the presentation before our PresentationU appointment, scheduled for next week.
Week 4 (February 26 - March 4, 2018)
Reality-Check
When we were first assigned this project, we assumed that our end goal would be to be able to identify different types of poker hands in an image of playing cards. However, after we met with our customers the first week, we realized that this was more of a stretch goal and would most likely not be achieved due to the lack of time. This week, we sat down and discussed our priorities. We all realized that we wanted to be able to create a fully functioning model, rather than one that had limitations. The limitations would be caused by the fact that we would not have enough time to train on all 52 cards. In order to achieve our goal of a complete model, we decided to change the game of poker to one of euchre, since it only requires 24 cards, rather than the whole deck. Though the game is different, the idea is the same: we want to be able to recognize different types of euchre hands in an image of playing cards. We also started to prepare for our Midterm Presentation. On Monday, we went to PresentationU to get feedback on our slides. We ran through our presentation in an informal manner with a student worker. At the end, she spoke about things she enjoyed regarding the slides and things she thought we could improve. She mentioned that she liked the way we all were able to clearly communicate what our project is about to someone that has no background in the Computer Science field. She was overall impressed with our presentation, but did suggest that we change a couple slides that were too wordy. We also realized that some of the requirements we first put on our website and our slides were outdated. We met on Sunday to go through the slides, in order to edit the more wordy slides and to update our requirements and schedule. As usual, we added more cards to our training set. Each week we set to add 60 images of 4 cards each. This week we added and annotated the Ace of Hearts, King of Diamonds, Queen of Diamonds, and Jack of Diamonds. Euchre only requires the 9, 10, Jack, Queen, King, and Ace of each suit, so we are getting good progress on our training set. We currently have half the suits of the Jack, Queen, King, and Ace cards. Now that we have switched games and updated how we plan to continue throughout the semester, we all feel much more comfortable and excited about the coming weeks and how our end project will play out!
Week 5 & 6 (March 5 - 18, 2018)
Showing Off
During Week 5, we presented our Midterm Presentation on Friday, March 9, 2018. For the days leading up to it, we mainly focused on editing and practicing our slides. Our main goal for the presentation was, of course, to meet all the requirements of what to communicate to the audience, but also to make the content as clear as possible. For example, the slides I presented were focused on conveying background information on neural nets - a topic which can get very complicated very quickly. Because of this, I spent the week figuring out the best way to keep the explanation as detailed, yet simple as possible. In addition, we pracitced our presentation many times. We wanted each other's feedback on the sections we were presenting, noting if there was anything we should modify. We also went through many dry runs for the sake of perfecting our delivery - many of us hadn't presented in a while and were a little rusty on speaking in front of an audience. Thanks to the large amount of practice, our midterm presentation went extremely smoothely. We also continued to collect images and annotations of cards throughout the two weeks. During Week 5, we completed our normal routine of 4 cards. However, we went over our schedule and realized - with the help of discussion with Dr. Piwowarski - that we should block out more time to focus on the actual application as opposed to just the training dataset collection. To accomplish this, we decided to double up on collecting the images and annotations over Spring Break, meaning we created a training dataset for 8 cards over the week, rather than 8.
Week 7 (March 19 - 25, 2018)
Starting A New Chapter
This week, we finally finished collecting our training set! This means, we finished taking all the images of the cards needed in Euchre and annotated each. Now that we are done, we have each taken and annotated roughly 1,440 images - 7,200 total! This is very exciting for al lof us because that was the most tedious portion of this project. We can now focus on the more exciting aspects! After collecting the training set, we decided we wanted to focus our energy on creating a website next. As our customers suggested, this would be for ease of testing on our part, as well as a much more user-friendly approach that a command line. So far, I have created the interface of our web application. I created a website using HTML and CSS, which is, at the moment, an outline of what we hope for it to include. The home page has a "browse" and "submit" button, allowing the user to upload an image and submit it to the server. There are a few other pages on the website, which will include more information about our project. Over break, Chelsea worked a lot on the server side of the application, so in the upcoming weeks I hope to help connect the front-end and back-end portions of the webpage! Here are some images of the website so far:
Week 8 (March 26 - April 1, 2018)
Mushing Everything Together
Since we finally compiled all 7,200+ of our images and annotations of the playing cards, Donald trained on the complete dataset this week. He explained that we were running into some problems with a few of the specific cards. I realized that a majority of the issues developed due to some of our images not uploading - this meant that our annotation (.xml) files were attempting to access images that did not exist. To fix this, a few of us had to re-upload our images onto OneDrive. In addition, there was another problem which was caused from non-matching file names in the annotation files. This was probably due to renaming the files after annotating so to fix this issue, I could have went through each file and typed in the correct file name, but decided to re-annotate the images instead. This felt a less tedious route to take and had the same result. After a few of us fixing the corrupted files, the training was finally a success! We met on Thursday to discuss the results. We were getting a pretty good percentage of recognition, but decided to take out images not from the specific Bicycle deck we had all been using to hopefully increase our accuracy even more. We will see how this turns out in the coming days. I also continued to work on the front-end of the webpage this week. Specifically, I added a few implementations that were purely design-related. For example, the title should now have a typewriter effect, allowing for a more interesting and eye-catching web application. Using Adam's suggestion, I also integrated the web application with this website. This can be seen by clicking the "Project" drop-down at the top right of this page. The very last option is "Web Application", where our website can be viewed. Stephanie created even more changes to the front-end, which look amazing! In addition, Chelsea is planning on integrating the front-end and back-end today. If we are able to have a connected model by tonight, I will work the rest of the week to add onto Stephanie's work of dealing with user interaction with the website. The team has been hard at work and it is exciting to be able to see the end result coming together!
Week 9 & 10 (April 2 - April 15, 2018)
Coming To An End
As discussed with Dr. Piwowarski, I have not been in town for the majority of the past two weeks. Unfortunately, this meant I was unable to contribute much to the project during this time. For the time that I was in town, I caught up on the wonderful progress of the rest of the team. In addition, we met with Dr. Piwowarski for a code review. He gave us great feedback, suggesting us to create more thorough documentation. Our program was initially based off a tutorial our customers recommended we use, meaning that there is quite a bit of code that is not what we wrote. It is important we document the code that we did write and explain why we made the changes that we did. Our practice presentation is this upcoming Friday, so Adam has started to make updates to our Midterm Presentation to be suitable for our Final Presentation. I plan on reviewing, editing, and adding to the PowerPoint tomorrow. We seem to be wrapping the project up and will be focusing our time on the more technical side of things, such as the documentation and presentation.
Week 11 (April 6 - April 23, 2018)
...And That's A Wrap!
During this past week, we had solely been focusing on wrapping things up, but more specifically, creating the final presentation. Adam had created a rough draft of the presentation. We all met up on Tuesday to look over it and make any necessary edits. We also assigned who would be presenting each slide, then ran through the presentation a few times. On Friday, we had our practice presentation in the Marksbury Theater (where the actual presentation will be held). Dr. Piwowarski listened to our talk in its entirety and gave some very useful feedback. On my end, he suggested I slow down when speaking, so I will be working more on that this week. I've thoroughly enjoyed this project and found the whole process exciting. I've also loved working with this team - everyone was wonderful to collaborate with and I couldn't have asked for better partners!