My experience in iOS Hackathon

This is my second hackathon, my first hackathon was on machine learning if you want to check out that article by following the below link
https://thangaayyanar.blogspot.com/2018/02/what-i-learned-from-machine-learning.html

So let's get started

First let us discuss about the idea of what we are trying to achieve in this hackathon.

From the above image you can able to know that we are going to recognize text from the image and use it to do find which field it is.

 we separated this idea into three modules

1. Identify the region
2. Recognize the text 
3. Field classification

Module I : Identify the region

• To identify the selected region we used Vision framework ( ML framework provided by apple to detect the object ).
• The vision framework give us the boundary of the text region ( i.e frame - x,y,width,height ).
•  Then using the above region we crop the selected region and pass it to the next module.

Module II : Recognize the text

• To recognize the text we tried lot of methods.
• First thing we go with Tesseract ( It is a Open Source OCR Engine ).
•  Tesseract take little bit time to configure and accuracy of the system is OK and it has one advantage we can train it with our data.
• Next we check out MLCore from google which has text detection .
• Configuration is not that hard if we followed the google docs and it's accuracy is better than Tesseract .
• When we integrate the MLCore we need to configure Firebase and make our project size pretty big (> 300 MB)
• Text detection support online and offline. we used offline version.
• Then we stick with google MLCore for accuracy.

Module III: Field classification

• To classify the Field we use Naive Bayes classifier
• We didn't use Machine learning model here because we don't have time.
• So we borrowed the code from a github developers who write a code how to do navie bayes in swift 
• The problem with this, we need to train it every time app start.

Now every module is done we integrated all the module. It worked not to our expected level and we figured where the problem is - it is Vision , i think we missed some thing so some text region are not cropped properly and time runs out

We submitted our project

Well we learnt lot of things from this hackathon

• Working with Machine learning in iOS
• We can able to convert model we trained in python to model that support in iOS done through CoreML tools
• Came across few new terms - CNN ( Convolutional Neural Network ), SVM, Classification algorithm
• Framework to develop model - keras,caffe,Tensorflow,scikit

I will drop few the link that i refer in this hackathon

Have a great day

Resources: 

• For Module I we followed this article - https://medium.com/flawless-app-stories/vision-in-ios-text-detection-and-tesseract-recognition-26bbcd735d8f
• For Tesseract integration we used this article - https://www.raywenderlich.com/163445/tesseract-ocr-tutorial-ios ( if you use tesseract add libc++ instead of libstdc++ )
• For MLCore Text detection - https://firebase.google.com/docs/ml-kit/ios/recognize-text
• For Navie bayes - https://github.com/fcanas/Bayes

Bonus Resources:

• Pre trained CoreML Models - https://github.com/likedan/Awesome-CoreML-Models
•  Handwritten training dataset - http://www.fki.inf.unibe.ch/databases/iam-handwriting-database
• Deep Learning tuts - https://github.com/priya-dwivedi/Deep-Learning
• Converting keras model to coreml model tuts - https://www.raywenderlich.com/181760/beginning-machine-learning-keras-core-ml