I helped orchestrate meetings with the customers at Lexmark. On top of that, I attended group meetings to figure out requirements, play story point poker, and determine the objective for the next several weeks. I also began researching the Quick Start: Distributed Training on the Oxford-IIIT Pets Dataset on Google Cloud tutorial found on github.com and tried to complete that to gain a better understanding of how to use neural networks (NNs) and train them. As I was running through this tutorial, I developed a batch script to aid my partners as they worked through the tutorial as well.
I got the project tuorial finally working thanks to Donald Venus's suggestions. Discovered that attempting to use Windows OS for this project is a horrible idea. Had I not used it, I may have been able to complete the tutorial much more quickly using the google cloud shell.
On top of that, I attended group meetings, since we had to schedule an extra one due to unforseen difficulties in the tutorial, to update the objective(s) for the next several weeks. We discussed requirement 2 and how to annotate the images and create the training and validation sets needed for the neural network.
I started development on a Python script to modify images and enable the team to take one image and generate several unique images for the training and validation sets. It could use some improvements like a better UI and modifying the brightness of the image and things of that nature, but it is still in early development. Now the real work begins with annotating the images.
Here is a link to the script's github repository: script
Spent some time developing the project presentation Powerpoint slides. Also updated the script that modifies the annotation .xml files created by the labelimg application. Specifically, I changed the file type in the path label to be set as a ".jpg" file. I also scheduled the group meeting times and discussed some tools we could use to make our lives easier when developing the project data and annotations. Finaslly, I resized the images I contributed to the data set from 4000x3000 to 300x300 and had to re-annotate the images. We found that the image sizes were too large and during training, they would consume ovver 100 GB of memory before the server would kill the training attempt. So, we realized thast we needed smaller image resolutions to prevent this from happening.
This week I added to the library of card images and annotations. On top of that, I discussed the project requirements and deliverables with the customers and whether they would prefer to have an entire library of cards or if they would prefer to have some type of interface. We came to the conclusion that we would be better served working on some type of web application for telling a user what type of hand we had. Chelsea Kuball had the great idea of making the most out of the smaller library and making it work for a card game called Euchre, which uses cards 9 - A. This would allow us to have a complete library for the game without taking 5000+ photos and annotations. It would leave room for other groups to add games to the application and more to the library. We met with the Presentation U! member and went over our project. They had glowing comments for our presentation and liked the amount of visuals in it. I created the design block diagrams for the project design. The two images were composed of the design for the web application and the design for the model and how it is developed.
The first thing I, and the rest of the group did was work on our midterm project presentation. We went over who would cover what parts of the project, what we would like improved and changed for our latest requirement updates, and performed dry runs of our presentations before the presentation occurred. On Friday, March 9th, the group presented the current project status and what we were hoping to do with the model going forward. This week(s) I continued adding to the image and annotation data sets and began looking into how to set up a web server using Flask and implementing a RESTful API. The team decided to double up on the amount of cards to add to the dat set since we would have more time during Spring break. I found some promising results for a server here and here. I tested a simple server from this site as well. While it allowed for simple post requests, our project needs to be able to send and receive images to provide the required functionality. So, I plan to continue researching how to implement such a server. The group is very excited about development so far.
I completed my round of contributions to all of the cards included in a Euchre deck. I also emailed the customers about project updates, how the presentation went, and their thoughts on implementing a recommendation system for the Euchre-based application and if it would be feasible to work on a neural network during the last few weeks of the semester. On top of those developments, I plan on digging into the server application to see if there are any possible improvements for the improving processing speed. It uses matplotlib, which I have some prior experience using for some CS460G assignments here at UK. I set up an initial testing plan, which was shared with the group so we could contribute our own testing ideas to each of the project components. The customers provided some feedback on the project and recommended that the group focus on completing the application before considering any work on a neural netwrok-based recommendation system.
I began the testing section of the groups' website adding all of the test cases featured on testing section of the website. On top of that, I had to make corrections to some faulty images and annotations. Some of the images simply did not upload so I had to remedy the issues for the images that I was responsible for. I also continued researching how to host a server using Docker and worked through a tutorial that uses Docker to host a Tensorflow model server. I ran into an issue with the group's current frozen model where the model has to be frozen with a "serve" tag in order for the Docker Virtual Machine/server to be able to recognize it as a viable model. It also looks for a directory structure where you have the root model directory and versions are contained in numbered subdirectories. So if I had a card_model directory as the root model directory, I would need version one and all the necessary files stored in a directory named "1" as a subdirectory of the card_model directory. Docker is useful because once you have a Docker image working, it makes serving the model much more portable and can be served on several different platforms. So, I will continue to research how to properly implement a Tensorflow model server using it. First, I will have to figure out how to freeze the model so that it is compatible with Docker.
After Chelsea succcessfully got a server working using Flask, I tried to clone her Github repository and reproduce the results. It took a little effort to get the directory tree required for the project, but I eventually got it working. I plan to deploy the server using the localhost tunneling application ngrok. I used it last semester in CS498 and it proved to be suffiecient for facilitating the deployment of a server. There was a function that I found was missing in one of the libraries and so I had to get that from Chelsea's machine. On top of that, I researched a way to publicly host the flask-based server and found that ngrok works fine for a demonstration so long as our server does not experience a heavy load. It probably would not scale well running on my 8 year old HP laptop. I only had to moify the mainpg.html file under the templates directory. By removing the "localhost:5000" text from the "localhost:5000/result" string for the action section of a form performing a 'POST', I was able to get the results to return to the client's PC screen even if it was not on a local machine.
I met with the group to discuss some concerning things I had noticed when setting up the server such as when submitting an image for labeling, the model would classify a single card object as two different card labels. After discussing the issues woth the group, we determined that there were a couple of things we could try including tweaking the model's learning rate, turning off a setting which mirrored the images when training in order to manufacture an extra training example, trying more accurate models, and a few other things. Another thing we discussed was checking the lowest precision card, which was the King of Diamonds, and seeing if our image quality for that card could use improvement. So, I spent time re-taking images and labeling them before submitting them to our image and annotation repository for training the model. On top of those things, I created a function for the server that would remove images it received for labeling requests so that it would not encumber the machine which hosted it. I, along with the rest of the team, also met with the professor for a code review to get feedback on areas to improve as far as documentation of the code goes and improving the maintainability of the code.
I attended a meeting to discuss the project presentation slides. I also added the final sections to the project webpage. On top of that, I created a github account that contains a directory tree for running the project application after cloning it to a machine. I also attended and participated in the group's practice presentation. On top of that, I added the last few sections to the group's project webpage.