Connect with us

AI

Using log analysis to drive experiments and win the AWS DeepRacer F1 ProAm Race

This is a guest post by Ray Goh, a tech executive at DBS Bank.  AWS DeepRacer is an autonomous 1/18th scale race car powered by reinforcement learning, and the AWS DeepRacer League is the world’s first global autonomous racing league. It’s a fun and easy way to get started with machine learning (ML), regardless of […]

Published

on

This is a guest post by Ray Goh, a tech executive at DBS Bank. 

AWS DeepRacer is an autonomous 1/18th scale race car powered by reinforcement learning, and the AWS DeepRacer League is the world’s first global autonomous racing league. It’s a fun and easy way to get started with machine learning (ML), regardless of skill or background. For companies, it’s also a powerful platform to facilitate teaching ML to employees at the enterprise level.

As part of our digital transformation journey at DBS Bank, we’re taking innovative steps to future-proof our workforce. We’ve partnered with AWS to bring the AWS DeepRacer League to DBS to train over 3,000 employees in AI and ML by the end of 2020. Thanks to the AWS DeepRacer virtual simulation and training environment, our employees can upgrade their skills and pick up new knowledge, even when they aren’t physically in the office. The ability to run private races also allows us to create our own racing league, where our employees can put their newly learned skills to the test.

Winning the F1 ProAm Race in May 2020

As an individual racer, I’ve been active in the AWS DeepRacer League since 2019. In May 2020, racers from around the world had the unique opportunity to pit their ML skills against F1 professionals in the AWS DeepRacer F1 ProAm Race. We trained our models on a replica of the F1 Spanish Grand Prix track, and the top 10 racers from the month-long, head-to-head qualifying race faced off against F1 professional drivers Daniel Ricciardo and Tatiana Calderon in a Grand Prix-style race. Watch the AWS DeepRacer ProAm series here.

After a challenging month of racing, I emerged as the champion in the F1 ProAm Race, beating fellow racers and the pro F1 drivers to the checkered flag! Looking back now, I attribute my win to having performed many experiments throughout the month of racing. Those experiments allowed me to continuously tweak and improve my model leading up to the final race. Behind those experiments are ideas that arose from data-driven insights through log analysis.

What is log analysis?

Log analysis is using a Jupyter notebook to analyze and debug models based on log data generated from the AWS DeepRacer simulation and training environment. With snippets of Python code, you can plot and visualize your model’s training performance through various graphs and heatmaps. I created several unique visualizations that ultimately helped me train a model that was fast and stable enough to win the F1 ProAm Race.

Figure 1 Log analysis visualizations

In this post, I share some of the visualizations I created and show how you can use Amazon SageMaker to spin up a notebook instance to perform log analysis using DeepRacer model training data.

If you’re already familiar with opening notebooks in a JupyterLab notebook application, you can simply clone my log analysis repository and skip directly to the log analysis section.

Amazon SageMaker notebook instances

An Amazon SageMaker notebook instance is a managed ML compute instance running the Jupyter notebook application. Amazon SageMaker manages the creation of the instance and its related resources, so we can focus on analyzing the data collected during training without worrying about provisioning Amazon Elastic Compute Cloud (Amazon EC2) or storage resources directly.

Using an Amazon SageMaker notebook instance for log analysis

One of the greatest benefits of using an Amazon SageMaker notebook instance to perform AWS DeepRacer log analysis is that Amazon SageMaker automatically installs Anaconda packages and libraries for common deep learning platforms on our behalf, including TensorFlow deep learning libraries. It also automatically attaches an ML storage volume to our notebook instance, which we can use as a persistent working storage to perform log analysis and retain our analysis artifacts.

Creating a notebook instance

To get started, create a notebook instance on the Amazon SageMaker console.

  1. On the Amazon SageMaker console, under Notebook, choose Notebook instances.
  2. Choose Create notebook instance.

  1. For Notebook instance name, enter a name (for example, DeepRacer-Log-Analysis).
  2. For Notebook instance type¸ choose your instance.

For AWS DeepRacer log analysis, the smallest instance type (ml.t2.medium) is usually sufficient.

  1. For Volume size in GB, enter your storage volume size. For this post, we enter 5.

When the notebook instance shows an InService status, we can open JupyterLab, the IDE for Jupyter notebooks.

  1. Locate your notebook instance and choose Open JupyterLab.

Cloning the log analysis repo from JupyterLab

From the JupyterLab IDE, we can easily clone a Git repository to use log analysis notebooks shared by the community. For example, I can clone my log analysis repository in seconds, using https://github.com/TheRayG/deepracer-log-analysis.git as the Clone URI.

After cloning the repository, we should see it appear in the folder structure on the left side of the JupyterLab IDE.

Downloading logs from the AWS DeepRacer console

To prepare the data that we want to analyze, we have to download our model training logs from the AWS DeepRacer console.

  1. On the AWS DeepRacer console, under Reinforcement learning, choose Your models.
  2. Choose the model to analyze.
  3. In the Training section, under Resources, choose Download Logs.

This downloads the training log files, which are packaged in a .tar.gz file.

Extracting the required log files for analysis

In this step, we complete the final configurations.

  1. Extract the RoboMaker and Amazon SageMaker log files from the .tar.gz package (found in the logs/training/ subdirectory).

  1. Upload the two log files into the /deepracer-log-analysis/logs folder in the JupyterLab IDE.

We’re now ready to open up our log analysis notebook to work its magic!

  1. Navigate to the /deepracer-log-analysis folder on the left side of the IDE and choose the .ipynb file to open the notebook.
  2. When opening the notebook, you may be prompted to provide a kernel. Choose a kernel that uses Python 3, such as conda_tensorflow_p36.

  1. Wait until the kernel status changes from Starting to Idle.
  2. Edit the notebook to specify the path and names of the two log files that we just uploaded.

To perform our visualizations, we use the simulation trace data from the RoboMaker log file and policy update data from the Amazon SageMaker log file. We parse the data in the notebook using pandas dataframes, which are two-dimensional labeled data structures like spreadsheets or SQL tables.

For the RoboMaker log file, we aggregate important information, such as minimum, maximum, and average progress and lap completion ratios for each iteration of training episodes.

For the Amazon SageMaker log file, we calculate the average entropy per epoch in each policy update iteration.

Performing visualizations

We can now run the notebook by choosing Run and Run All Cells in JupyterLab. My log analysis notebook contains numerous markdown descriptions and comments to explain what each cell does. In this section, I highlight some of the visualizations from that notebook and explain some of the thought processes behind them.

Visualizing the performance envelope of the model

A common question asked by beginners of AWS DeepRacer is, “If two models are trained for the same amount of time using the same reward function and hyperparameters, why do they have different lap times when I evaluate them?”

The following visualization is a great way to explain it; it shows the frequency of performance to lap time in seconds.

I use this to illustrate the performance envelope of my model. We can show the relative probability of the model achieving various lap times by plotting a histogram of lap times achieved by the model during training. We can also work out statistically the average and best-case lap times that we can expect from the model. I’ve noticed that the lap times of the model during training resembles a normal distribution, so I use the -2 and -3 Std Dev markers to show the potential best-case lap times for the model, albeit with just 2.275% (-2 SD) and 0.135% (-3 SD) chance of occurring respectively. By understanding the likelihood of the model achieving a given lap time and comparing that to leaderboard times, I can gauge if I should continue cloning and tweaking the model, or abandon it and start fresh with a different approach.

Identifying potential model checkpoints for race submission

When training many different models for a race, racers commonly ask, “Which model would give me the highest chance of winning a virtual race?”

To answer that question, I plot the top quartile (p25) lap times vs. iterations from the training data, which identifies potential models for race submission. This scatter plot also allows me to identify potential trade-offs between speed (dots with very fast lap times) and stability (dense cluster of dots for a particular iteration). From the following diagram, I would choose models from the three highlighted iterations for race submission.

Identifying convergence and gauging consistency

As racers gain experience with model training, they start paying attention to convergence in their models. Simply put, convergence in the AWS DeepRacer context is when a model is performing close to its best (in terms of average lap progress), and further training may harm its performance or make it overfit, such that it only does well for that track in a very specific simulation environment, but not in other tracks or in a physical AWS DeepRacer car. That begs the following questions: “How do I tell when the model has converged?” and “How consistent is my model after it has converged?”

To aid in visualizing convergence, I overlay the entropy information from the Amazon SageMaker policy training logs over the usual plots for rewards and progress.

Entropy is a measure of the amount of randomness in our reinforcement learning neural network. At the beginning of model training, entropy is high, because our neural network is updated mostly based on random actions as the car explores the track.

Over time, with more experiences gained from actions and rewards at various parts of the track, the car starts to exploit this information and takes less random actions.

The thinking behind this is that, as rewards and progress increase, the entropy value should decrease. When rewards and progress plateau, the entropy loss should also flatten out. Therefore, I use entropy as an additional indicator for convergence.

To gauge the consistency of my model, I also plot the percentage of lap completions per iteration during training. When the model is capable of completing laps, the percentage of completed laps should creep up in subsequent iterations, until around the point of convergence, when the percentage value should plateau too. See the following plot.

The model training process is probabilistic because the reinforcement learning agent incorporates entropy to explore the environment. To smooth out the effects of the probabilistic model in my visualization, I use a simple moving average over three iterations for each of my plotted metrics.

Identifying inefficiencies in driving behavior

When racers have a competitive model, they may start to wonder, “Are there sections of the track where the car is driving inefficiently? What are the sections where I can encourage the car to speed up?”

In pursuit of answering these questions, I designed a visualization that shows the average speed and steering angle of the car measured at every waypoint along the track. This allows me to see how the model is negotiating the track, because from this plot, you can see the rate at which the model is speeding up or slowing down as it travels through the waypoints. The following visualization shows the deviation of the optimal racing line (orange) from the track centerline (blue).

You can also see how the model adjusts its steering angle as it negotiates turns. What I love about the following visualization is that it allows me to see clearly at which point after a long straight the model starts to brake before entering into a turn. It also helps me visualize if a model is accelerating quickly enough upon exiting a turn.

Identifying track sections to adjust actions and rewards

Although speed is the primary performance criteria in a time trial race, stability is also important in an object avoidance or head-to-head race. Because time penalties for going off-track impact race position, it’s very important to find the right balance between speed and stability. Even if the model can negotiate the track well, top racers are also asking, “Is the car over- or under-steering at any of the turns? Which turn should I focus on optimizing in subsequent experiments?”

By plotting a heatmap of rewards over the track, you can easily see how consistently we reward the model at various parts of the track. A thin band in the heatmap reflects very consistent rewards, while a sparse scattering of dots brings attention to the parts of the track where the model has trouble getting rewards. For my reward function, this usually highlights the turns at which the model is over- or under-steering.

For example, in the highlighted parts of the preceding plot, the model isn’t consistently going around those turns according to the racing line that I’m rewarding for. It’s actually over-steering as it exits Turn 3 (around waypoint 62), and under-steering around the other two highlighted turns. Tweaking the action space may help (in the case of under-steering, lowering the speed at high steering angles). Interestingly, the lap completion rate of the model can increase substantially with such minor tweaks, without sacrificing lap times!

Experiment, Experiment, Experiment

For the F1 ProAm Race that in May 2020, I planned to do two experiments per day (at least 60 experiments total) to try out different reward strategies and racing lines. I could iterate quickly while focusing on incremental improvements by using log analysis to surface insights from the training data.

For example, the following plot helped me answer the question “Is the car going to go as fast as possible through the entire lap?” by showing where the car uses 0-degree and highest speeds.

Cleaning up

To save on ML compute costs, when you’re done with log analysis, you can stop the notebook instance without deleting it. The notebook, data, and log files are still retained as long as you don’t delete the notebook instance. A stopped instance still incurs cost for the provisioned ML storage. But you can always restart the instance later to continue working on the notebook.

When you no longer need the notebook or data, you can permanently delete the instance, which also deletes the attached ML storage volume, so that you no longer incur its related ML storage cost.

For pricing details for Amazon SageMaker notebook instances, see Amazon SageMaker Pricing.

Conclusion

The visualizations I shared with you in this post helped me win the May 2020 F1 ProAm Race against other top racers and F1 pros, so it’s my hope that by sharing these ideas with the community, others can benefit and learn from them too.

Together as a community of practice, we can help to accelerate learning for everyone and raise the bar for the AI/ML community in general!

You can start training your own model and improve it through log analysis by signing in to the AWS DeepRacer console.


About the Author

Ray Goh is a Tech executive who leads Agile Teams in the delivery of FX Trading & Digital Solutions at DBS Bank. He is a passionate Cloud advocate with deep interest in Voice and Serverless technology, and has 8 AWS Certifications under his belt. He is also active in the DeepRacer (a Machine Learning autonomous model car) community. Obsessed with home automation, he owns close to 20 Alexa-enabled devices at home and in the car.

Source: https://aws.amazon.com/blogs/machine-learning/using-log-analysis-to-drive-experiments-and-win-the-aws-deepracer-f1-proam-race/

AI

Arcanum makes Hungarian heritage accessible with Amazon Rekognition

Arcanum specializes in digitizing Hungarian language content, including newspapers, books, maps, and art. With over 30 years of experience, Arcanum serves more than 30,000 global subscribers with access to Hungarian culture, history, and heritage. Amazon Rekognition Solutions Architects worked with Arcanum to add highly scalable image analysis to Hungaricana, a free service provided by Arcanum, […]

Published

on

Arcanum specializes in digitizing Hungarian language content, including newspapers, books, maps, and art. With over 30 years of experience, Arcanum serves more than 30,000 global subscribers with access to Hungarian culture, history, and heritage.

Amazon Rekognition Solutions Architects worked with Arcanum to add highly scalable image analysis to Hungaricana, a free service provided by Arcanum, which enables you to search and explore Hungarian cultural heritage, including 600,000 faces over 500,000 images. For example, you can find historical works by author Mór Jókai or photos on topics like weddings. The Arcanum team chose Amazon Rekognition to free valuable staff from time and cost-intensive manual labeling, and improved label accuracy to make 200,000 previously unsearchable images (approximately 40% of image inventory), available to users.

Amazon Rekognition makes it easy to add image and video analysis to your applications using highly scalable machine learning (ML) technology that requires no previous ML expertise to use. Amazon Rekognition also provides highly accurate facial recognition and facial search capabilities to detect, analyze, and compare faces.

Arcanum uses this facial recognition feature in their image database services to help you find particular people in Arcanum’s articles. This post discusses their challenges and why they chose Amazon Rekognition as their solution.

Automated image labeling challenges

Arcanum dedicated a team of three people to start tagging and labeling content for Hungaricana. The team quickly learned that they would need to invest more than 3 months of time-consuming and repetitive human labor to provide accurate search capabilities to their customers. Considering the size of the team and scope of the existing project, Arcanum needed a better solution that would automate image and object labelling at scale.

Automated image labeling solutions

To speed up and automate image labeling, Arcanum turned to Amazon Rekognition to enable users to search photos by keywords (for example, type of historic event, place name, or a person relevant to Hungarian history).

For the Hungaricana project, preprocessing all the images was challenging. Arcanum ran a TensorFlow face search across all 28 million pages on a machine with 8 GPUs in their own offices to extract only faces from images.

The following screenshot shows what an extract looks like (image provided by Arcanum Database Ltd).

The images containing only faces are sent to Amazon Rekognition, invoking the IndexFaces operation to add a face to the collection. For each face that is detected in the specified face collection, Amazon Rekognition extracts facial features into a feature vector and stores it in an Amazon Aurora database. Amazon Rekognition uses feature vectors when it performs face match and search operations using the SearchFaces and SearchFacesByImage operations.

The image preprocessing helped create a very efficient and cost-effective way to index faces. The following diagram summarizes the preprocessing workflow.

As for the web application, the workflow starts with a Hungaricana user making a face search request. The following diagram illustrates the application workflow.

The workflow includes the following steps:

  1. The user requests a facial match by uploading the image. The web request is automatically distributed by the Elastic Load Balancer to the webserver fleet.
  2. Amazon Elastic Compute Cloud (Amazon EC2) powers application servers that handle the user request.
  3. The uploaded image is stored in Amazon Simple Storage Service (Amazon S3).
  4. Amazon Rekognition indexes the face and runs SearchFaces to look for a face similar to the new face ID.
  5. The output of the search face by image operation is stored in Amazon ElastiCache, a fully managed in-memory data store.
  6. The metadata of the indexed faces are stored in an Aurora relational database built for the cloud.
  7. The resulting face thumbnails are served to the customer via the fast content-delivery network (CDN) service Amazon CloudFront.

Experimenting and live testing Hungaricana

During our test of Hungaricana, the application performed extremely well. The searches not only correctly identified people, but also provided links to all publications and sources in Arcanum’s privately owned database where found faces are present. For example, the following screenshot shows the result of the famous composer and pianist Franz Liszt.

The application provided 42 pages of 6×4 results. The results are capped to 1,000. The 100% scores are the confidence scores returned by Amazon Rekognition and are rounded up to whole numbers.

The application of Hungaricana has always promptly, and with a high degree of certainty, presented results and links to all corresponding publications.

Business results

By introducing Amazon Rekognition into their workflow, Arcanum enabled a better customer experience, including building family trees, searching for historical figures, and researching historical places and events.

The concept of face searching using artificial intelligence certainly isn’t new. But Hungaricana uses it in a very creative, unique way.

Amazon Rekognition allowed Arcanum to realize three distinct advantages:

  • Time savings – The time to market speed increased dramatically. Now, instead of spending several months of intense manual labor to label all the images, the company can do this job in a few days. Before, basic labeling on 150,000 images took months for three people to complete.
  • Cost savings – Arcanum saved around $15,000 on the Hungaricana project. Before using Amazon Rekognition, there was no automation, so a human workforce had to scan all the images. Now, employees can shift their focus to other high-value tasks.
  • Improved accuracy – Users now have a much better experience regarding hit rates. Since Arcanum started using Amazon Rekognition, the number of hits has doubled. Before, out of 500,000 images, about 200,000 weren’t searchable. But with Amazon Rekognition, search is now possible for all 500,000 images.

 “Amazon Rekognition made Hungarian culture, history, and heritage more accessible to the world,” says Előd Biszak, Arcanum CEO. “It has made research a lot easier for customers building family trees, searching for historical figures, and researching historical places and events. We cannot wait to see what the future of artificial intelligence has to offer to enrich our content further.”

Conclusion

In this post, you learned how to add highly scalable face and image analysis to an enterprise-level image gallery to improve label accuracy, reduce costs, and save time.

You can test Amazon Rekognition features such as facial analysis, face comparison, or celebrity recognition on images specific to your use case on the Amazon Rekognition console.

For video presentations and tutorials, see Getting Started with Amazon Rekognition. For more information about Amazon Rekognition, see Amazon Rekognition Documentation.


About the Authors

Siniša Mikašinović is a Senior Solutions Architect at AWS Luxembourg, covering Central and Eastern Europe—a region full of opportunities, talented and innovative developers, ISVs, and startups. He helps customers adopt AWS services as well as acquire new skills, learn best practices, and succeed globally with the power of AWS. His areas of expertise are Game Tech and Microsoft on AWS. Siniša is a PowerShell enthusiast, a gamer, and a father of a small and very loud boy. He flies under the flags of Croatia and Serbia.

Cameron Peron is Senior Marketing Manager for AWS Amazon Rekognition and the AWS AI/ML community. He evangelizes how AI/ML innovation solves complex challenges facing community, enterprise, and startups alike. Out of the office, he enjoys staying active with kettlebell-sport, spending time with his family and friends, and is an avid fan of Euro-league basketball.

Source: https://aws.amazon.com/blogs/machine-learning/arcanum-makes-hungarian-heritage-accessible-with-amazon-rekognition/

Continue Reading

AI

Arcanum makes Hungarian heritage accessible with Amazon Rekognition

Arcanum specializes in digitizing Hungarian language content, including newspapers, books, maps, and art. With over 30 years of experience, Arcanum serves more than 30,000 global subscribers with access to Hungarian culture, history, and heritage. Amazon Rekognition Solutions Architects worked with Arcanum to add highly scalable image analysis to Hungaricana, a free service provided by Arcanum, […]

Published

on

Arcanum specializes in digitizing Hungarian language content, including newspapers, books, maps, and art. With over 30 years of experience, Arcanum serves more than 30,000 global subscribers with access to Hungarian culture, history, and heritage.

Amazon Rekognition Solutions Architects worked with Arcanum to add highly scalable image analysis to Hungaricana, a free service provided by Arcanum, which enables you to search and explore Hungarian cultural heritage, including 600,000 faces over 500,000 images. For example, you can find historical works by author Mór Jókai or photos on topics like weddings. The Arcanum team chose Amazon Rekognition to free valuable staff from time and cost-intensive manual labeling, and improved label accuracy to make 200,000 previously unsearchable images (approximately 40% of image inventory), available to users.

Amazon Rekognition makes it easy to add image and video analysis to your applications using highly scalable machine learning (ML) technology that requires no previous ML expertise to use. Amazon Rekognition also provides highly accurate facial recognition and facial search capabilities to detect, analyze, and compare faces.

Arcanum uses this facial recognition feature in their image database services to help you find particular people in Arcanum’s articles. This post discusses their challenges and why they chose Amazon Rekognition as their solution.

Automated image labeling challenges

Arcanum dedicated a team of three people to start tagging and labeling content for Hungaricana. The team quickly learned that they would need to invest more than 3 months of time-consuming and repetitive human labor to provide accurate search capabilities to their customers. Considering the size of the team and scope of the existing project, Arcanum needed a better solution that would automate image and object labelling at scale.

Automated image labeling solutions

To speed up and automate image labeling, Arcanum turned to Amazon Rekognition to enable users to search photos by keywords (for example, type of historic event, place name, or a person relevant to Hungarian history).

For the Hungaricana project, preprocessing all the images was challenging. Arcanum ran a TensorFlow face search across all 28 million pages on a machine with 8 GPUs in their own offices to extract only faces from images.

The following screenshot shows what an extract looks like (image provided by Arcanum Database Ltd).

The images containing only faces are sent to Amazon Rekognition, invoking the IndexFaces operation to add a face to the collection. For each face that is detected in the specified face collection, Amazon Rekognition extracts facial features into a feature vector and stores it in an Amazon Aurora database. Amazon Rekognition uses feature vectors when it performs face match and search operations using the SearchFaces and SearchFacesByImage operations.

The image preprocessing helped create a very efficient and cost-effective way to index faces. The following diagram summarizes the preprocessing workflow.

As for the web application, the workflow starts with a Hungaricana user making a face search request. The following diagram illustrates the application workflow.

The workflow includes the following steps:

  1. The user requests a facial match by uploading the image. The web request is automatically distributed by the Elastic Load Balancer to the webserver fleet.
  2. Amazon Elastic Compute Cloud (Amazon EC2) powers application servers that handle the user request.
  3. The uploaded image is stored in Amazon Simple Storage Service (Amazon S3).
  4. Amazon Rekognition indexes the face and runs SearchFaces to look for a face similar to the new face ID.
  5. The output of the search face by image operation is stored in Amazon ElastiCache, a fully managed in-memory data store.
  6. The metadata of the indexed faces are stored in an Aurora relational database built for the cloud.
  7. The resulting face thumbnails are served to the customer via the fast content-delivery network (CDN) service Amazon CloudFront.

Experimenting and live testing Hungaricana

During our test of Hungaricana, the application performed extremely well. The searches not only correctly identified people, but also provided links to all publications and sources in Arcanum’s privately owned database where found faces are present. For example, the following screenshot shows the result of the famous composer and pianist Franz Liszt.

The application provided 42 pages of 6×4 results. The results are capped to 1,000. The 100% scores are the confidence scores returned by Amazon Rekognition and are rounded up to whole numbers.

The application of Hungaricana has always promptly, and with a high degree of certainty, presented results and links to all corresponding publications.

Business results

By introducing Amazon Rekognition into their workflow, Arcanum enabled a better customer experience, including building family trees, searching for historical figures, and researching historical places and events.

The concept of face searching using artificial intelligence certainly isn’t new. But Hungaricana uses it in a very creative, unique way.

Amazon Rekognition allowed Arcanum to realize three distinct advantages:

  • Time savings – The time to market speed increased dramatically. Now, instead of spending several months of intense manual labor to label all the images, the company can do this job in a few days. Before, basic labeling on 150,000 images took months for three people to complete.
  • Cost savings – Arcanum saved around $15,000 on the Hungaricana project. Before using Amazon Rekognition, there was no automation, so a human workforce had to scan all the images. Now, employees can shift their focus to other high-value tasks.
  • Improved accuracy – Users now have a much better experience regarding hit rates. Since Arcanum started using Amazon Rekognition, the number of hits has doubled. Before, out of 500,000 images, about 200,000 weren’t searchable. But with Amazon Rekognition, search is now possible for all 500,000 images.

 “Amazon Rekognition made Hungarian culture, history, and heritage more accessible to the world,” says Előd Biszak, Arcanum CEO. “It has made research a lot easier for customers building family trees, searching for historical figures, and researching historical places and events. We cannot wait to see what the future of artificial intelligence has to offer to enrich our content further.”

Conclusion

In this post, you learned how to add highly scalable face and image analysis to an enterprise-level image gallery to improve label accuracy, reduce costs, and save time.

You can test Amazon Rekognition features such as facial analysis, face comparison, or celebrity recognition on images specific to your use case on the Amazon Rekognition console.

For video presentations and tutorials, see Getting Started with Amazon Rekognition. For more information about Amazon Rekognition, see Amazon Rekognition Documentation.


About the Authors

Siniša Mikašinović is a Senior Solutions Architect at AWS Luxembourg, covering Central and Eastern Europe—a region full of opportunities, talented and innovative developers, ISVs, and startups. He helps customers adopt AWS services as well as acquire new skills, learn best practices, and succeed globally with the power of AWS. His areas of expertise are Game Tech and Microsoft on AWS. Siniša is a PowerShell enthusiast, a gamer, and a father of a small and very loud boy. He flies under the flags of Croatia and Serbia.

Cameron Peron is Senior Marketing Manager for AWS Amazon Rekognition and the AWS AI/ML community. He evangelizes how AI/ML innovation solves complex challenges facing community, enterprise, and startups alike. Out of the office, he enjoys staying active with kettlebell-sport, spending time with his family and friends, and is an avid fan of Euro-league basketball.

Source: https://aws.amazon.com/blogs/machine-learning/arcanum-makes-hungarian-heritage-accessible-with-amazon-rekognition/

Continue Reading

AI

Arcanum makes Hungarian heritage accessible with Amazon Rekognition

Arcanum specializes in digitizing Hungarian language content, including newspapers, books, maps, and art. With over 30 years of experience, Arcanum serves more than 30,000 global subscribers with access to Hungarian culture, history, and heritage. Amazon Rekognition Solutions Architects worked with Arcanum to add highly scalable image analysis to Hungaricana, a free service provided by Arcanum, […]

Published

on

Arcanum specializes in digitizing Hungarian language content, including newspapers, books, maps, and art. With over 30 years of experience, Arcanum serves more than 30,000 global subscribers with access to Hungarian culture, history, and heritage.

Amazon Rekognition Solutions Architects worked with Arcanum to add highly scalable image analysis to Hungaricana, a free service provided by Arcanum, which enables you to search and explore Hungarian cultural heritage, including 600,000 faces over 500,000 images. For example, you can find historical works by author Mór Jókai or photos on topics like weddings. The Arcanum team chose Amazon Rekognition to free valuable staff from time and cost-intensive manual labeling, and improved label accuracy to make 200,000 previously unsearchable images (approximately 40% of image inventory), available to users.

Amazon Rekognition makes it easy to add image and video analysis to your applications using highly scalable machine learning (ML) technology that requires no previous ML expertise to use. Amazon Rekognition also provides highly accurate facial recognition and facial search capabilities to detect, analyze, and compare faces.

Arcanum uses this facial recognition feature in their image database services to help you find particular people in Arcanum’s articles. This post discusses their challenges and why they chose Amazon Rekognition as their solution.

Automated image labeling challenges

Arcanum dedicated a team of three people to start tagging and labeling content for Hungaricana. The team quickly learned that they would need to invest more than 3 months of time-consuming and repetitive human labor to provide accurate search capabilities to their customers. Considering the size of the team and scope of the existing project, Arcanum needed a better solution that would automate image and object labelling at scale.

Automated image labeling solutions

To speed up and automate image labeling, Arcanum turned to Amazon Rekognition to enable users to search photos by keywords (for example, type of historic event, place name, or a person relevant to Hungarian history).

For the Hungaricana project, preprocessing all the images was challenging. Arcanum ran a TensorFlow face search across all 28 million pages on a machine with 8 GPUs in their own offices to extract only faces from images.

The following screenshot shows what an extract looks like (image provided by Arcanum Database Ltd).

The images containing only faces are sent to Amazon Rekognition, invoking the IndexFaces operation to add a face to the collection. For each face that is detected in the specified face collection, Amazon Rekognition extracts facial features into a feature vector and stores it in an Amazon Aurora database. Amazon Rekognition uses feature vectors when it performs face match and search operations using the SearchFaces and SearchFacesByImage operations.

The image preprocessing helped create a very efficient and cost-effective way to index faces. The following diagram summarizes the preprocessing workflow.

As for the web application, the workflow starts with a Hungaricana user making a face search request. The following diagram illustrates the application workflow.

The workflow includes the following steps:

  1. The user requests a facial match by uploading the image. The web request is automatically distributed by the Elastic Load Balancer to the webserver fleet.
  2. Amazon Elastic Compute Cloud (Amazon EC2) powers application servers that handle the user request.
  3. The uploaded image is stored in Amazon Simple Storage Service (Amazon S3).
  4. Amazon Rekognition indexes the face and runs SearchFaces to look for a face similar to the new face ID.
  5. The output of the search face by image operation is stored in Amazon ElastiCache, a fully managed in-memory data store.
  6. The metadata of the indexed faces are stored in an Aurora relational database built for the cloud.
  7. The resulting face thumbnails are served to the customer via the fast content-delivery network (CDN) service Amazon CloudFront.

Experimenting and live testing Hungaricana

During our test of Hungaricana, the application performed extremely well. The searches not only correctly identified people, but also provided links to all publications and sources in Arcanum’s privately owned database where found faces are present. For example, the following screenshot shows the result of the famous composer and pianist Franz Liszt.

The application provided 42 pages of 6×4 results. The results are capped to 1,000. The 100% scores are the confidence scores returned by Amazon Rekognition and are rounded up to whole numbers.

The application of Hungaricana has always promptly, and with a high degree of certainty, presented results and links to all corresponding publications.

Business results

By introducing Amazon Rekognition into their workflow, Arcanum enabled a better customer experience, including building family trees, searching for historical figures, and researching historical places and events.

The concept of face searching using artificial intelligence certainly isn’t new. But Hungaricana uses it in a very creative, unique way.

Amazon Rekognition allowed Arcanum to realize three distinct advantages:

  • Time savings – The time to market speed increased dramatically. Now, instead of spending several months of intense manual labor to label all the images, the company can do this job in a few days. Before, basic labeling on 150,000 images took months for three people to complete.
  • Cost savings – Arcanum saved around $15,000 on the Hungaricana project. Before using Amazon Rekognition, there was no automation, so a human workforce had to scan all the images. Now, employees can shift their focus to other high-value tasks.
  • Improved accuracy – Users now have a much better experience regarding hit rates. Since Arcanum started using Amazon Rekognition, the number of hits has doubled. Before, out of 500,000 images, about 200,000 weren’t searchable. But with Amazon Rekognition, search is now possible for all 500,000 images.

 “Amazon Rekognition made Hungarian culture, history, and heritage more accessible to the world,” says Előd Biszak, Arcanum CEO. “It has made research a lot easier for customers building family trees, searching for historical figures, and researching historical places and events. We cannot wait to see what the future of artificial intelligence has to offer to enrich our content further.”

Conclusion

In this post, you learned how to add highly scalable face and image analysis to an enterprise-level image gallery to improve label accuracy, reduce costs, and save time.

You can test Amazon Rekognition features such as facial analysis, face comparison, or celebrity recognition on images specific to your use case on the Amazon Rekognition console.

For video presentations and tutorials, see Getting Started with Amazon Rekognition. For more information about Amazon Rekognition, see Amazon Rekognition Documentation.


About the Authors

Siniša Mikašinović is a Senior Solutions Architect at AWS Luxembourg, covering Central and Eastern Europe—a region full of opportunities, talented and innovative developers, ISVs, and startups. He helps customers adopt AWS services as well as acquire new skills, learn best practices, and succeed globally with the power of AWS. His areas of expertise are Game Tech and Microsoft on AWS. Siniša is a PowerShell enthusiast, a gamer, and a father of a small and very loud boy. He flies under the flags of Croatia and Serbia.

Cameron Peron is Senior Marketing Manager for AWS Amazon Rekognition and the AWS AI/ML community. He evangelizes how AI/ML innovation solves complex challenges facing community, enterprise, and startups alike. Out of the office, he enjoys staying active with kettlebell-sport, spending time with his family and friends, and is an avid fan of Euro-league basketball.

Source: https://aws.amazon.com/blogs/machine-learning/arcanum-makes-hungarian-heritage-accessible-with-amazon-rekognition/

Continue Reading
AI10 hours ago

Arcanum makes Hungarian heritage accessible with Amazon Rekognition

AI10 hours ago

Arcanum makes Hungarian heritage accessible with Amazon Rekognition

AI10 hours ago

Arcanum makes Hungarian heritage accessible with Amazon Rekognition

AI10 hours ago

Arcanum makes Hungarian heritage accessible with Amazon Rekognition

AI10 hours ago

Arcanum makes Hungarian heritage accessible with Amazon Rekognition

AI10 hours ago

Arcanum makes Hungarian heritage accessible with Amazon Rekognition

AI10 hours ago

Arcanum makes Hungarian heritage accessible with Amazon Rekognition

AI10 hours ago

Arcanum makes Hungarian heritage accessible with Amazon Rekognition

AI10 hours ago

Arcanum makes Hungarian heritage accessible with Amazon Rekognition

AI10 hours ago

Arcanum makes Hungarian heritage accessible with Amazon Rekognition

AI10 hours ago

Arcanum makes Hungarian heritage accessible with Amazon Rekognition

AI10 hours ago

Arcanum makes Hungarian heritage accessible with Amazon Rekognition

AI14 hours ago

Pros and Cons of using cloud platforms for building chatbots

AI14 hours ago

From Knowledge Databases To Knowledge Graphs And Conversational AI

AI14 hours ago

Model selection with cross-validation: A quest for an elite model

AI15 hours ago

Celebrating 10 Years of Innovation, Excellence, and Trust

AI1 day ago

Executive Interview: Brian Gattoni, CTO, Cybersecurity & Infrastructure Security Agency 

AI1 day ago

Making Use Of AI Ethics Tuning Knobs In AI Autonomous Cars 

AI1 day ago

Application of AI to IT Service Ops by IBM and ServiceNow Exemplifies a Trend 

AI1 day ago

Testing Finds Automated Driver Assistance Systems to be Unreliable 

Trending