Connect with us

AI

Activity detection on a live video stream with Amazon SageMaker

Live video streams are continuously generated across industries including media and entertainment, retail, and many more. Live events like sports, music, news, and other special events are broadcast for viewers on TV and other online streaming platforms. AWS customers increasingly rely on machine learning (ML) to generate actionable insights in real time and deliver an […]

Published

on

Live video streams are continuously generated across industries including media and entertainment, retail, and many more. Live events like sports, music, news, and other special events are broadcast for viewers on TV and other online streaming platforms.

AWS customers increasingly rely on machine learning (ML) to generate actionable insights in real time and deliver an enhanced viewing experience or timely alert notifications. For example, AWS Sports explains how leagues, broadcasters, and partners can train teams, engage fans, and transform the business of sports with ML.

Amazon Rekognition makes it easy to add image and video analysis to your applications using proven, highly scalable, deep learning technology that requires no ML expertise to use. With Amazon Rekognition, you can identify objects, people, text, scenes, and some pre-defined activities in videos.

However, if you want to use your own video activity dataset and your own model or algorithm, you can use Amazon SageMaker. Amazon SageMaker is a fully managed service that allows you to build, train, and deploy ML models quickly. Amazon SageMaker removes the heavy lifting from each step of the ML process to make it easier to develop high-quality models.

In this post, you use Amazon SageMaker to automatically detect activities from a custom dataset in a live video stream.

The large volume of live video streams generated needs to be stored, processed in real time, and reviewed by a human team at low latency and low cost. Such pipelines include additional processing steps if specific activities are automatically detected in a video segment, such as penalty kicks in football triggering the generation of highlight clips for viewer replay.

ML inference on a live video stream can have the following challenges:

  • Input source – The live stream video input must be segmented and made available in a data store for consumption with negligible latency.
  • Large payload size – A video segment of 10 seconds can range from 1–15 MB. This is significantly larger than a typical tabular data record.
  • Frame sampling – Video frames must be extracted at a sampling rate generally lesser than the original video FPS (frames per second) efficiently and without a loss in output accuracy.
  • Frame preprocessing – Video frames must be preprocessed to reduce image size and resolution without a loss in output accuracy. They also need to be normalized across all the images used in training.
  • External dependencies – External libraries like opencv and ffmpeg with different license types are generally used for image processing.
  • 3D model network – Video models are typically 3D networks with an additional temporal dimension to image networks. They work on 5D input (batch size, time, RGB channel, height, width) and require a large volume of annotated input videos.
  • Low latency, low cost – The output ML inference needs to be generated within an acceptable latency and at a low cost to demonstrate value.

This post describes two phases of a ML lifecycle. In the first phase, you build, train, and deploy a 3D video classification model on Amazon SageMaker. You fine-tune a pre-trained model with a ResNet50 backbone by transfer learning on another dataset and test inference on a sample video segment. The pre-trained model from a well-known model zoo reduces the need for large volumes of annotated input videos and can be adapted for tasks in another domain.

In the second phase, you deploy the fine-tuned model on an Amazon SageMaker endpoint in a production context of an end-to-end solution with a live video stream. The live video stream is simulated by a sample video in a continuous playback loop with AWS Elemental MediaLive. Video segments from the live stream are delivered to Amazon Simple Storage Service (Amazon S3), which invokes the Amazon SageMaker endpoint for inference. The inference payload is the Amazon S3 URI of the video segment. The use of the Amazon S3 URI is efficient because it eliminates the need to serialize and deserialize a large video frame payload over a REST API and transfers the responsibility of ingesting the video to the endpoint. The endpoint inference performs frame sampling and pre-processing followed by video segment classification, and outputs the result to Amazon DynamoDB.

Finally, the solution performance is summarized with respect to latency, throughput, and cost.

For the complete code associated with this post, see the GitHub repo.

Building, training, and deploying an activity detection model with Amazon SageMaker

In this section, you use Amazon SageMaker to build, train, and deploy an activity detection model in a development environment. The following diagram depicts the Amazon SageMaker ML pipeline used to perform the end-to-end model development. The training and inference Docker images are available for any deep learning framework of choice, like TensorFlow, Pytorch, and MXNet. You use the “bring your own script” capability of Amazon SageMaker to provide a custom entrypoint script for training and inference. For this use case, you use transfer learning to fine-tune a pretrained model from a model zoo with another dataset loaded in Amazon S3. You use model artifacts from the training job to deploy an endpoint with automatic scaling, which dynamically adjusts the number of instances in response to changes in the invocation workload.

For this use case, you use two popular activity detection datasets: Kinetics400 and UCF101. Dataset size is a big factor in the performance of deep learning models. Kinetics400 has 306,245 short trimmed videos from 400 action categories. However, you may not have such a high volume of annotated videos in other domains. Training a deep learning model on small datasets from scratch may lead to severe overfitting, which is why you need transfer learning.

UCF101 has 13,320 videos from 101 action categories. For this post, we use UCF101 as a dataset from another domain for transfer learning. The pre-trained model is fine-tuned by replacing the last classification (dense) layer to the number of classes in the UCF101 dataset. The final test inference is run on new sample videos from Pexels that were unavailable during the training phase. Similarly, you can train good models on activity videos from your own domain without large annotated datasets and with less computing resource utilization.

We chose Apache MXNet as the deep learning framework for this use case because of the availability of the GluonCV toolkit. GluonCV provides implementations of state-of-the-art deep learning algorithms in computer vision. It features training scripts that reproduce state-of-the-art results reported in the latest research papers, a model zoo with a large set of pre-trained models, carefully designed APIs, and easy-to-understand implementations and community support. The action recognition model zoo contains multiple pre-trained models with the Kinetics400 dataset. The following graph shows the inference throughputs vs. validation accuracy and device memory footprint for each of them.

I3D (Inflated 3D Networks) and SlowFast are 3D video classification networks with varying trade-offs between accuracy and efficiency. For this post, we chose the Inflated 3D model (I3D) with ResNet50 backbone trained on the Kinetics400 dataset. It uses 3D convolution to learn spatiotemporal information directly from videos. The input to this model is of the form (N x C x T x H x W), where N is the batch size, C is the number of colour channels, T is the number of frames in the video segment, H is the height of the video frame, and W is the width of the video frame.

Both model training and inference require efficient and convenient slicing methods for video preprocessing. Decord provides such methods based on a thin wrapper on top of hardware accelerated video decoders. We use Decord as a video reader, and use VideoClsCustom as a custom GluonCV video data loader for pre-processing.

We launch the training script mode with the Amazon SageMaker MXNet training toolkit and the AWS Deep Learning container for training on Apache MXNet. The training process includes the following steps for the entire dataset:

  • Frame sampling and center cropping
  • Frame normalization with mean and standard deviation across all ImageNet images
  • Transfer learning on the pre-trained model with a new dataset

After you have a trained model artifact, you can include it in a Docker container that runs your inference script and deploys to an Amazon SageMaker endpoint. Inference is launched with the Amazon SageMaker MXNet inference toolkit and the AWS Deep Learning container for inference on Apache MXNet. The inference process includes the following steps for the video segment payload:

  • Pre-processing (similar to training)
  • Activity classification

We use the Amazon Elastic Compute Cloud (Amazon EC2) G4 instance for our Amazon SageMaker endpoint hosting. G4 instances provide the latest generation NVIDIA T4 GPUs, AWS custom Intel Cascade Lake CPUs, up to 100 Gbps of networking throughput, and up to 1.8 TB of local NVMe storage. G4 instances are optimized for computer vision application deployments like image classification and object detection. For more information about inference benchmarks, see NVIDIA Data Center Deep Learning Product Performance.

We test the model inference deployed as an Amazon SageMaker endpoint using the following video examples (included in the code repository).

Output activity : TennisSwing

Output activity : Skiing

Note that the UCF101 dataset does not have a snowboarding activity class. The most similar activity present in the dataset is skiing and the model predicts skiing when evaluated with a snowboarding video.

You can find the complete Amazon SageMaker Jupyter notebook example with transfer learning and inference on the GitHub repo. After the model is trained, deployed to an Amazon SageMaker endpoint, and tested with the sample videos in the development environment, you can deploy it in a production context of an end-to-end solution.

Deploying the solution with AWS CloudFormation

In this step, you deploy the end-to-end activity detection solution using an AWS CloudFormation template. After a successful deployment, you use the model to automatically detect an activity in a video segment from a live video stream. The following diagram depicts the roles of the AWS services in the solution.

The AWS Elemental MediaLive livestreaming channel that is created from a sample video file in S3 bucket is used to demonstrate the real-time architecture. You can use other live streaming services as well that are capable of delivering live video segments to S3.

  1. AWS Elemental MediaLive sends live video with HTTP Live Streaming (HLS) and regularly generates fragments of equal length (10 seconds) as .ts files and an index file that contains references of the fragmented files as a .m3u8 file in a S3 bucket.
  2. An upload of each video .ts fragment into the S3 bucket triggers a lambda function.
  3. The lambda function simply invokes a SageMaker endpoint for activity detection with the S3 URI of the video fragment as the REST API payload input.
  4. The SageMaker inference container reads the video from S3, pre-processes it, detects an activity and saves the prediction results to an Amazon DynamoDB table.

For this post, we use the sample Skiing People video to create the MediaLive live stream channel (also included in the code repository). In addition, the deployed model is the I3D model with Resnet50 backbone fine-tuned with the UCF101 dataset, as explained in the previous section. Autoscaling is enabled for the Amazon SageMaker endpoint to adjust the number of instances based on the actual workload.

The end-to-end solution example is provided in the GitHub repo. You can also use your own sample video and fine-tuned model.

There is a cost associated with deploying and running the solution, as mentioned in the Cost Estimation section of this post. Make sure to delete the CloudFormation stack if you no longer need it. The steps to delete the solution are detailed in the section Cleaning up.

You can deploy the solution by launching the CloudFormation stack:

The solution is deployed in the us-east-1 Region. For instructions on changing your Region, see the GitHub repo. After you launch the CloudFormation stack, you can update the parameters for your environments or leave the defaults. All their descriptions are provided when you launch it. Don’t create or select any other options in the configuration. In addition, don’t create or select an AWS Identity and Access Management (IAM) role on the Permissions tab. At the final page of the CloudFormation creation process, you need to select the two check-boxes under Capabilities. To create the AWS resources associated with the solution, you should have permissions to create CloudFormation stacks.

After you launch the stack, make sure that the root stack with its nested sub-stacks are successfully created by viewing the stack on the AWS CloudFormation console.

Wait until all the stacks are successfully created before you proceed to the next session. It takes 15–20 minutes to deploy the architecture.

The root CloudFormation stack ActivityDetectionCFN is mainly used to call the other nested sub-stacks and create the corresponding AWS resources.

The sub-stacks have the root stack name as their prefix. This is mainly to show that they are nested. In addition, the AWS Region name and account ID are added as suffices in the live stream S3 bucket name to avoid naming conflicts.

After the solution is successfully deployed, the following AWS resources are created:

  • S3 bucket – The bucket activity-detection-livestream-bucket-us-east-1-<account-id> stores video segments from the live video stream
  • MediaLive channel – The channel activity-detection-channel creates video segments from the live video stream and saves them to the S3 bucket
  • Lambda function – The function activity-detection-lambda invokes the Amazon SageMaker endpoint to detect activities from each video segment
  • SageMaker endpoint – The endpoint activity-detection-endpoint loads a video segment, detects an activity, and saves the results into a DynamoDB table
  • DynamoDB table – The table activity-detection-table stores the prediction results from the endpoint

Other supporting resources, like IAM roles, are also created to provide permissions to their respective resources.

Using the Solution

After the solution is successfully deployed, it’s time to test the model with a live video stream. You can start the channel on the MediaLive console.

Wait for the channel state to change to Running. When the channel state changes to Running, .ts-formatted video segments are saved into the live stream S3 bucket.

Only the most recent 21 video segments are kept in the S3 bucket to save storage. For more information about increasing the number of segments, see the GitHub repo.

Each .ts-formatted video upload triggers a Lambda function. The function invokes the Amazon SageMaker endpoint with an Amazon S3 URI to the .ts video as the payload. The deployed model predicts the activity and saves the results to the DynamoDB table.

Evaluating performance results

The Amazon SageMaker endpoint instance in this post is a ml.g4dn.2xlarge with 8 vCPU and 1 Nvidia T4 Tensor Core GPU. The solution generates one video segment every 10 seconds at 30 FPS. Therefore, there are six endpoint requests per minute (RPM). The overhead latency in video segment creation from the live stream up to delivery in Amazon S3 is approximately 1 second.

Amazon SageMaker supports automatic scaling for your hosted models. Autoscaling dynamically adjusts the number of instances provisioned for a model in response to changes in your workload. You can simulate a load test to determine the throughput of the system and the scaling policy for autoscaling in Amazon SageMaker. For the activity detection model in this post, a single ml.g4dn.2xlarge instance hosting the endpoint can handle 600 RPM with a latency of 300 milliseconds and with less than 5% number of HTTP errors. With a SAFETY_FACTOR = 0.5, the autoscaling target metric can be set as SageMakerVariantInvocationsPerInstance = 600 * 0.5 = 300 RPM.

For more information on load testing, see Load testing your autoscaling configuration.

Observe the differences between two scenarios under the same increasing load up to 1,700 invocations per minute in a period of 15 minutes. The BEFORE scenario is for a single instance without autoscaling, and the AFTER scenario is after two additional instances are launched by autoscaling.

The following graphs report the number of invocations and invocations per instance per minute. In the BEFORE scenario, with a single instance, both lines merge. In the AFTER scenario, the load pattern and Invocations (the blue line) are similar, but InvocationsPerInstance (the red line) is lower because the endpoint has scaled out.

The following graphs report average ModelLatency per minute. In the BEFORE scenario, the average latency remains under 300 milliseconds for the first 3 minutes of the test. The average ModelLatency exceeds 1 second after the endpoint gets more than 600 RPS and goes up to 8 seconds during peak load. In the AFTER scenario, the endpoint has already scaled to three instances and the average ModelLatency per minute remains close to 300 milliseconds.

The following graphs report the errors from the endpoint. In the BEFORE scenario, as the load exceeds 300 RPS, the endpoint starts to error out for some requests and increases as the number of invocations increase. In the AFTER scenario, the endpoint has scaled out already and reports no errors.

In the AFTER scenario, at peak load, average CPU utilization across instances is near maximum (100 % * 8 vcpu = 800 %) and average GPU utilization across instances is about 35%. To improve GPU utilization, additional effort can be made to optimize the CPU intensive pre-processing.

Cost Estimation

The following table summarizes the cost estimation for this solution. The cost includes the per hour pricing for one Amazon SageMaker endpoint instance of type ml.g4dn.2xlarge in the us-east-1 Region. If autoscaling is applied, the cost per hour includes the cost of the number of active instances. Also, MediaLive channel pricing is governed by the input and output configuration for the channel.

Component Details  Pricing (N.Virginia) Cost per hour
Elemental Media Live Single pipeline, On-Demand, SD $0.036 / hr input + $0.8496 / hr output

$ 0.8856

S3 30 mb per minute $0.023 per gb $ 0.0414
Lambda Only invoke sm, minimum config (128 mb) $0.0000002083 per 100 ms

$ 0.00007

SageMaker endpoint (ml.g4dn.2xlarge) single instance $1.053 per hour $1.053
DynamoDB $1.25 per million write request units $ 0.0015

Cleaning up

If you no longer need the solution, complete the following steps to properly delete all the associated AWS resources:

  1. On the MediaLive console, stop the live stream channel.
  2. On the Amazon S3 console, delete the S3 bucket you created earlier.
  3. On the AWS CloudFormation console, delete the root ActivityDetectionCFN This also removes all nested sub-stacks.

Conclusion

In this post, you used Amazon SageMaker to automatically detect activity in a simulated live video stream. You used a pre-trained model from the gluon-cv model zoo and Apache MXNet framework for transfer learning on another dataset. You also used Amazon SageMaker inference to preprocess and classify a video segment delivered to Amazon S3. Finally, you evaluated the solution respect to model latency, system throughput, and cost.


About the Authors

Hasan Poonawala is a Machine Learning Specialist Solution Architect at AWS, based in London, UK. Hasan helps customers design and deploy machine learning applications in production on AWS. He is passionate about the use of machine learning to solve business problems across various industries. In his spare time, Hasan loves to explore nature outdoors and spend time with friends and family.

Tesfagabir Meharizghi is a Data Scientist at the Amazon ML Solutions Lab where he works with customers across different verticals accelerate their use of machine learning and AWS cloud services to solve their business challenges. Outside of work, he enjoys spending time with his family and reading books.

Source: https://aws.amazon.com/blogs/machine-learning/activity-detection-on-a-live-video-stream-with-amazon-sagemaker/

AI

Are Chatbots Vulnerable? Best Practices to Ensure Chatbots Security

Published

on

Rebecca James
credit IT Security Guru

A simple answer is a Yes! Chatbots are vulnerable. Some specific threats and vulnerabilities risk chatbots security and prove them a wrong choice for usage. With the advancement in technology, hackers can now easily target the hidden infrastructure of a chatbot.

The chatbot’s framework has an opportunity for the attackers ready to inject the malicious codes or commands that might unlock the secured data of the customers and your business. However, the extent of the attack’s complexity and success might depend on the messaging platform’s security.

Are you thinking about how chatbots are being exposed to attacks? Well! Hackers are now highly advanced. They attack the chatbots in two ways, i.e., either by social engineering attack or by technical attacks.

  • An evil bot can impersonate a legal user by using backup data of the possibly targeted victims by social engineering attack. All such data is collected from various sources like the dark web and social media platforms. Sometimes they use both sources to gain access to some other user’s data by a bot providing such services.
  • The second attack is technical. Here also attackers can turn themself into evil bots who exchange messages with the other bots. The purpose is to look for some vulnerabilities in the target’s profile that can be later exploited. It can eventually lead to the compromise of the entire framework that protects the data and can ultimately lead to data theft.

To ensure chatbots security, the bot creators must ensure that all the security processes are in place and are responsible for restoring the architecture. The data flow via the chatbot system should also be encrypted both in transit and rest.

To further aid you in chatbot security, this article discusses five best practices to ensure chatbots security. So, let’s read on.

The following mentioned below are some of the best practices to ensure the security of chatbots.

It’s always feared that data in transit can be spoofed or tampered with the sophistication of cybercriminals’ technology and smartness. It’s essential to implement end-to-end encryption to ensure that your entire conversation remains secured. It means that by encryption, you can prevent any third person other than the sender and the receiver from peeping into your messages.

Encryption importance can’t be neglected in the cyber world, and undoubtedly the chatbot designers are adapting this method to make sure that their chatbot security is right on the point. For more robust encryption, consider using business VPNs that encrypt your internet traffic and messages. With a VPN, you can also prevent the threats and vulnerabilities associated with chatbots.

1. 8 Proven Ways to Use Chatbots for Marketing (with Real Examples)

2. How to Use Texthero to Prepare a Text-based Dataset for Your NLP Project

3. 5 Top Tips For Human-Centred Chatbot Design

4. Chatbot Conference Online

Moreover, it’s a crucial feature of other chat services like WhatsApp and other giant tech developers. They are anxious to guarantee security via encryption even when there’s strict surveillance by the government. Such encryption is to fulfill the legal principles of the GDPR that says that companies should adopt measures to encrypt the users’ data.

User identity authentication is a process that verifies if the user is having secure and valid credentials like the username and password. The login credentials are exchanged for having a secure authentication token used during the complete user session. If you haven’t, then you should try out this method for boosting user security.

Authentication timeouts are another way to ensure your chatbots security. This method is more common in banks as the token can be used for the predetermined time.

Moreover, two-factor authentication is yet another method to prove user identity. Users are asked to verify identity either by a text message or email, depending on the way they’ve chosen. It also helps in the authorization process as it permits access to the right person and ensures that information isn’t mishandled or breached.

The self-destructive message features open another way for enhancing chatbot security. This option comes in handy when the user provides their personally identifiable information. Such information can pose a serious threat to user privacy and should be destroyed or deleted within a set period. This method is handier when you’re associated with backing or any other financial chatbots.

By using secure protocols, you can also ensure chatbots security. Every security system, by default, has the HTTPS protocol installed in it. If you aren’t an IT specialist, you can also identify it when you view the search bar’s URL. As long as your data is being transferred via HTTPS protocol and encrypted connections, TLS and SSL, your data is secured from vulnerabilities and different types of cyber-attacks.

Thus, make sure to use secure protocols for enhanced security. Remember that when Chatbots are new, the coding and system used to protect it is the same as the existing HIMs. They interconnect with their security systems and have more than one encryption layer to protect their users’ security.

Do you know what the most significant security vulnerability that’s challenging to combat is? Wondering? Well! It’s none other than human error. User behavior must be resolved using commercial applications because they might continue to believe that the systems are flawed.

No doubt that an unprecedented number of users label the significance of digital security, but still, humans are the most vulnerable in the system. Chatbot security continues to be a real big problem until the problem of user errors comes to an end. And this needs education on various forms of digital technology, including chatbots.

Here the customers aren’t the ones who are to be blamed. Like customers, employees can make a mistake, and they do make most of the time. To prevent this, the chatbot developers should form a defined strategy, including the IT experts, and train them on the system’s safe use. Doing so enhances the team’s skillset and allows them to engage with the chatbot system confidently.

However, clients can’t be educated like the employees. But at least you can provide them a detailed road map of securely interacting with the system. It might involve other professionals who can successfully engage customers and educate them on the right way to interact with the chatbots.

Several emerging technologies are keen to play a vital role in protecting the chatbots against threats and vulnerabilities in the upcoming time, among all the most potent method behavior analytics and Artificial Intelligence developments.

  • User Behavioral Analytics: It’s a process that uses applications to study the patterns of user behavior. It enables them to implement complex algorithms and statistical analysis to detect any abnormal behavior that possibly represents a security threat. Analytical tools are quite common and powerful; thus, this methodology can become a fundamental component of the chatbot system.
  • Developments in AI: Artificial technology is a two-end sword that offers benefits and threats simultaneously. But, as AI is predicted to fulfill its potential, it will provide an extra security level to the systems. It is mainly because of its ability to wipe a large amount of data for abnormalities that recognizes security breaches and threats.

The Bottom Line

Security concerns have always been there with new technologies and bring new threats and vulnerabilities with them. Although chatbots are an emerging technology, the security practices that stand behind them are present for a long time and are effective. Chatbots are the innovative development of the current era, and emerging technologies like AI will transform the way businesses might interact with the customers and ensure their security.

Source: https://chatbotslife.com/are-chatbots-vulnerable-best-practices-to-ensure-chatbots-security-d301b9f6ce17?source=rss—-a49517e4c30b—4

Continue Reading

AI

Best Technology Stacks For Mobile App Development

Published

on

What’s the Best Tech Stack for Mobile App Development? Read To Know

Which is the Best Tech Stack for Mobile Application Development? Kotlin, React Native, Ionic, Xamarin, Objective-C, Swift, JAVA… Which One?

Image Source: Google

Technology Stack for smartphones is like what blood is for the human body. Without a technology stack, it is hard even to imagine smartphones. Having a smartphone in uncountable hands is rising exponentially. For tech pundits, this is one unmissable aspect of our digital experience wherein tech stack is as critical as ROI.

The riveting experience for a successful mobile app predominantly depends on technology stacks.

The unbiased selection of mobile apps development language facilitates developers to build smooth, functional, efficient apps. They help businesses tone down the costs, focus on revenue-generation opportunities. Most importantly, it provides customers with jaw-dropping amazement, giving a reason to have it installed on the indispensable gadget in present times.

In today’s time, when there are over 5 million apps globally, and by all conscience, these are whopping no.s and going to push the smartphone industry further. But now you could see mobile app development every ‘nook and corner.’ But the fact is not who provides what but understanding the behavioural pattern of users.

So the pertinent question is, which is the ideal tech stack to use for mobile app development?

In native mobile app development, all toolkits, mobile apps development language, and the SDK are supported and provided by operating system vendors. Native app development thus allows developers to build apps compatible with specific OS environments; it is suitable for device-specific hardware and software. Hence it renders optimized performance using the latest technology. However, since Android & iOS imparts — — a unique platform for development, businesses have to develop multiple mobile apps for each platform.

1. Waz

2. Pokemon Go

3. Lyft

1.Java: The popularity of JAVA still makes it one of the official programming languages for android app development until the introduction of Kotlin. Java itself is at the core of the Android OS. Many of us even see the logo of Java when the device reboots. However, contradictions with Oracle (which owns the license to Java) made Google shift to open-source Java SDK for versions starting from Android 7.0 Nougat

2.Kotlin: According to Google I/O conference in 2019- Kotlin is the officially supported language for Android app development. It is entirely based on Java but has a few additions which make it simpler and easier to work.

1. 8 Proven Ways to Use Chatbots for Marketing (with Real Examples)

2. How to Use Texthero to Prepare a Text-based Dataset for Your NLP Project

3. 5 Top Tips For Human-Centred Chatbot Design

4. Chatbot Conference Online

It’s my gut feeling like other developers to say that Kotlin is simply better. It has a leaner, more straightforward and concise code than open-cell Java, and several other advantages about handling null-pointer exceptions and more productive coding.

HERE’S A Programming Illustration Defining the CONCISENESS OF KOTLIN CODE

public class Address {

private String street;

private int streetNumber;

private String postCode;

private String city;

private Country country;

public Address(String street, int streetNumber, String postCode, String city, Country country) {

this.street = street;

this.streetNumber = streetNumber;

this.postCode = postCode;

this.city = city;

this.country = country;

}

@Override

public boolean equals(Object o) {

if (this == o) return true;

if (o == null || getClass() != o.getClass()) return false;

Address address = (Address) o;

if (streetNumber != address.streetNumber) return false;

if (!street.equals(address.street)) return false;

if (!postCode.equals(address.postCode)) return false;

if (!city.equals(address.city)) return false;

return country == address.country;

}

@Override

public int hashCode() {

int result = street.hashCode();

result = 31 * result + streetNumber;

result = 31 * result + postCode.hashCode();

result = 31 * result + city.hashCode();

result = 31 * result + (country != null ? country.hashCode() : 0);

return result;

}

@Override

public String toString() {

return “Address{“ +

“street=’” + street + ‘\’’ +

“, streetNumber=” + streetNumber +

“, postCode=’” + postCode + ‘\’’ +

“, city=’” + city + ‘\’’ +

“, country=” + country +

‘}’;

}

public String getStreet() {

return street;

}

public void setStreet(String street) {

this.street = street;

}

public int getStreetNumber() {

return streetNumber;

}

public void setStreetNumber(int streetNumber) {

this.streetNumber = streetNumber;

}

public String getPostCode() {

return postCode;

}

public void setPostCode(String postCode) {

this.postCode = postCode;

}

public String getCity() {

return city;

}

public void setCity(String city) {

this.city = city;

}

public Country getCountry() {

return country;

}

public void setCountry(Country country) {

this.country = country;

}

}

class Address(street:String, streetNumber:Int, postCode:String, city:String, country:Country) {

var street: String

var streetNumber:Int = 0

var postCode:String

var city: String

var country:Country

init{

this.street = street

this.streetNumber = streetNumber

this.postCode = postCode

this.city = city

this.country = country

}

public override fun equals(o:Any):Boolean {

if (this === o) return true

if (o == null || javaClass != o.javaClass) return false

Val address = o as Address

if (streetNumber != address.streetNumber) return false

if (street != address.street) return false

if (postCode != address.postCode) return false

if (city != address.city) return false

return country === address.country

}

public override fun hashCode():Int {

val result = street.hashCode()

result = 31 * result + streetNumber

result = 31 * result + postCode.hashCode()

result = 31 * result + city.hashCode()

result = 31 * result + (if (country != null) country.hashCode() else 0)

return result

}

public override fun toString():String {

return (“Address{“ +

“street=’” + street + ‘\’’.toString() +

“, streetNumber=” + streetNumber +

“, postCode=’” + postCode + ‘\’’.toString() +

“, city=’” + city + ‘\’’.toString() +

“, country=” + country +

‘}’.toString())

}

}

I’d say KOTLIN IS THE BEST FIND FOR ANDROID APP DEVELOPMENT.Google has dug deeper with some plans ahead since announcing it as an official language. Moreover, it signals Google’s first steps in moving away from the Java ecosystem, which is imminent, considering its recent adventures with Flutter and the upcoming Fuchsia OS.

Objective C is the same for iOS what Java is for Android. Objective-C, a superset of the C programming language( with objective -oriented capabilities and dynamic run time) initially used to build the core of iOS operating system across the Apple devices. However, Apple soon started using swift, which diminishes the importance of Objective -C in comparison to previous compilations.

Apple introduced Swift as an alternative to Objective-C in late 2015, and it has since been continued to be the primary language for iOS app development.Swift is more functional than Objective-C, less prone to errors, dynamic libraries help reduce the size and app without ever compromising performance.

Now, you would remember the comparison we’ve done with Java and kotlin. In iOS, objective-C is much older than swift with much more complicated syntax. Giving cringeworthy feel to beginners to get started with Objective-C.

Image Source: Google

THIS IS WHAT YOU DO WHEN INITIALIZING AN ARRAY IN OBJECTIVE-C:

NSMutableArray * array =[[NSMutableArray alloc] init];

NOW LOOK AT HOW THE SAME THING IS DONE IN SWIFT:

var array =[Int]()

SWIFT IS MUCH MORE ` WHAT WE’VE COVERED HERE.

In cross-platform app development, developers build a single mobile app that can be used on multiple OS platforms. It is made possible by creating an app with a shared common codebase, adapted to various platforms.

Image Source: Google

Popular Cross-platform apps:

  1. Instagram
  2. Skype
  3. LinkedIN

React Native is a mobile app development framework based on JavaScript. It is used and supported by one of the biggest social media platforms- Facebook. In cross-platform apps built using React Native, the application logic is coded in JavaScript, whereas its UI is entirely native. This blog about building a React Native app is worth reading if you want to know why its stakes are higher.

Xamarin is a Microsoft-supported cross-platform mobile app development tool that uses the C# programming language. Using Xamarin, developers can build mobile apps for multiple platforms, sharing over 90% of the same code.

TypeScript is a superset of JavaScript, and is a statically-typed programming language supported by Microsoft. TypeScript can be used along with the React Native framework to make full use of its error detection features when writing code for react components.

In Hybrid mobile app development, developers build web apps using HTML, CSS & JavaScript and then wrap the code in a native shell. It allows the app to be deployed as a regular app, with functionality at a level between a fully native app and a website rendered(web browser).

Image Source: Google
  1. Untappd
  2. Amazon App Store
  3. Evernote

Apache Cordova is an open-source hybrid mobile app development framework that uses JavaScript for logic operations and while HTML5 & CSS3 for rendering. PhoneGap is a commercialized, free, and open-source distribution of Apache Cordova owned by Adobe. The PhoneGap platform was developed to deliver non-proprietary, free, and open-source app development solutions powered by the web.

Ionic is a hybrid app development framework based on AngularJS. Similar to other hybrid platforms, it uses HTML, CSS & JavaScript to build mobile apps. Ionic is primarily focused on the front-end UI experience and integrates well with frameworks such as Angular, Vue, and ReactJS.

To summarize, there are 3 types of mobile apps- Native mobile apps, Cross-platform mobile apps, and Hybrid mobile apps; each offers unique technologies, frameworks, and tools of their own. I have enlisted here the best mobile app technology stacks you could use for mobile app development.

The technologies, tools, and frameworks mentioned here are used in some of the most successful apps. With support from an expert, a well-established mobile app development company, that may give much-needed impetus in the dynamic mobile app development world.

Source: https://chatbotslife.com/best-technology-stacks-for-mobile-app-development-6fed70b62778?source=rss—-a49517e4c30b—4

Continue Reading

AI

Arcanum makes Hungarian heritage accessible with Amazon Rekognition

Arcanum specializes in digitizing Hungarian language content, including newspapers, books, maps, and art. With over 30 years of experience, Arcanum serves more than 30,000 global subscribers with access to Hungarian culture, history, and heritage. Amazon Rekognition Solutions Architects worked with Arcanum to add highly scalable image analysis to Hungaricana, a free service provided by Arcanum, […]

Published

on

Arcanum specializes in digitizing Hungarian language content, including newspapers, books, maps, and art. With over 30 years of experience, Arcanum serves more than 30,000 global subscribers with access to Hungarian culture, history, and heritage.

Amazon Rekognition Solutions Architects worked with Arcanum to add highly scalable image analysis to Hungaricana, a free service provided by Arcanum, which enables you to search and explore Hungarian cultural heritage, including 600,000 faces over 500,000 images. For example, you can find historical works by author Mór Jókai or photos on topics like weddings. The Arcanum team chose Amazon Rekognition to free valuable staff from time and cost-intensive manual labeling, and improved label accuracy to make 200,000 previously unsearchable images (approximately 40% of image inventory), available to users.

Amazon Rekognition makes it easy to add image and video analysis to your applications using highly scalable machine learning (ML) technology that requires no previous ML expertise to use. Amazon Rekognition also provides highly accurate facial recognition and facial search capabilities to detect, analyze, and compare faces.

Arcanum uses this facial recognition feature in their image database services to help you find particular people in Arcanum’s articles. This post discusses their challenges and why they chose Amazon Rekognition as their solution.

Automated image labeling challenges

Arcanum dedicated a team of three people to start tagging and labeling content for Hungaricana. The team quickly learned that they would need to invest more than 3 months of time-consuming and repetitive human labor to provide accurate search capabilities to their customers. Considering the size of the team and scope of the existing project, Arcanum needed a better solution that would automate image and object labelling at scale.

Automated image labeling solutions

To speed up and automate image labeling, Arcanum turned to Amazon Rekognition to enable users to search photos by keywords (for example, type of historic event, place name, or a person relevant to Hungarian history).

For the Hungaricana project, preprocessing all the images was challenging. Arcanum ran a TensorFlow face search across all 28 million pages on a machine with 8 GPUs in their own offices to extract only faces from images.

The following screenshot shows what an extract looks like (image provided by Arcanum Database Ltd).

The images containing only faces are sent to Amazon Rekognition, invoking the IndexFaces operation to add a face to the collection. For each face that is detected in the specified face collection, Amazon Rekognition extracts facial features into a feature vector and stores it in an Amazon Aurora database. Amazon Rekognition uses feature vectors when it performs face match and search operations using the SearchFaces and SearchFacesByImage operations.

The image preprocessing helped create a very efficient and cost-effective way to index faces. The following diagram summarizes the preprocessing workflow.

As for the web application, the workflow starts with a Hungaricana user making a face search request. The following diagram illustrates the application workflow.

The workflow includes the following steps:

  1. The user requests a facial match by uploading the image. The web request is automatically distributed by the Elastic Load Balancer to the webserver fleet.
  2. Amazon Elastic Compute Cloud (Amazon EC2) powers application servers that handle the user request.
  3. The uploaded image is stored in Amazon Simple Storage Service (Amazon S3).
  4. Amazon Rekognition indexes the face and runs SearchFaces to look for a face similar to the new face ID.
  5. The output of the search face by image operation is stored in Amazon ElastiCache, a fully managed in-memory data store.
  6. The metadata of the indexed faces are stored in an Aurora relational database built for the cloud.
  7. The resulting face thumbnails are served to the customer via the fast content-delivery network (CDN) service Amazon CloudFront.

Experimenting and live testing Hungaricana

During our test of Hungaricana, the application performed extremely well. The searches not only correctly identified people, but also provided links to all publications and sources in Arcanum’s privately owned database where found faces are present. For example, the following screenshot shows the result of the famous composer and pianist Franz Liszt.

The application provided 42 pages of 6×4 results. The results are capped to 1,000. The 100% scores are the confidence scores returned by Amazon Rekognition and are rounded up to whole numbers.

The application of Hungaricana has always promptly, and with a high degree of certainty, presented results and links to all corresponding publications.

Business results

By introducing Amazon Rekognition into their workflow, Arcanum enabled a better customer experience, including building family trees, searching for historical figures, and researching historical places and events.

The concept of face searching using artificial intelligence certainly isn’t new. But Hungaricana uses it in a very creative, unique way.

Amazon Rekognition allowed Arcanum to realize three distinct advantages:

  • Time savings – The time to market speed increased dramatically. Now, instead of spending several months of intense manual labor to label all the images, the company can do this job in a few days. Before, basic labeling on 150,000 images took months for three people to complete.
  • Cost savings – Arcanum saved around $15,000 on the Hungaricana project. Before using Amazon Rekognition, there was no automation, so a human workforce had to scan all the images. Now, employees can shift their focus to other high-value tasks.
  • Improved accuracy – Users now have a much better experience regarding hit rates. Since Arcanum started using Amazon Rekognition, the number of hits has doubled. Before, out of 500,000 images, about 200,000 weren’t searchable. But with Amazon Rekognition, search is now possible for all 500,000 images.

 “Amazon Rekognition made Hungarian culture, history, and heritage more accessible to the world,” says Előd Biszak, Arcanum CEO. “It has made research a lot easier for customers building family trees, searching for historical figures, and researching historical places and events. We cannot wait to see what the future of artificial intelligence has to offer to enrich our content further.”

Conclusion

In this post, you learned how to add highly scalable face and image analysis to an enterprise-level image gallery to improve label accuracy, reduce costs, and save time.

You can test Amazon Rekognition features such as facial analysis, face comparison, or celebrity recognition on images specific to your use case on the Amazon Rekognition console.

For video presentations and tutorials, see Getting Started with Amazon Rekognition. For more information about Amazon Rekognition, see Amazon Rekognition Documentation.


About the Authors

Siniša Mikašinović is a Senior Solutions Architect at AWS Luxembourg, covering Central and Eastern Europe—a region full of opportunities, talented and innovative developers, ISVs, and startups. He helps customers adopt AWS services as well as acquire new skills, learn best practices, and succeed globally with the power of AWS. His areas of expertise are Game Tech and Microsoft on AWS. Siniša is a PowerShell enthusiast, a gamer, and a father of a small and very loud boy. He flies under the flags of Croatia and Serbia.

Cameron Peron is Senior Marketing Manager for AWS Amazon Rekognition and the AWS AI/ML community. He evangelizes how AI/ML innovation solves complex challenges facing community, enterprise, and startups alike. Out of the office, he enjoys staying active with kettlebell-sport, spending time with his family and friends, and is an avid fan of Euro-league basketball.

Source: https://aws.amazon.com/blogs/machine-learning/arcanum-makes-hungarian-heritage-accessible-with-amazon-rekognition/

Continue Reading
AI3 hours ago

Are Chatbots Vulnerable? Best Practices to Ensure Chatbots Security

AI3 hours ago

Best Technology Stacks For Mobile App Development

AI23 hours ago

Arcanum makes Hungarian heritage accessible with Amazon Rekognition

AI23 hours ago

Arcanum makes Hungarian heritage accessible with Amazon Rekognition

AI23 hours ago

Arcanum makes Hungarian heritage accessible with Amazon Rekognition

AI23 hours ago

Arcanum makes Hungarian heritage accessible with Amazon Rekognition

AI23 hours ago

Arcanum makes Hungarian heritage accessible with Amazon Rekognition

AI23 hours ago

Arcanum makes Hungarian heritage accessible with Amazon Rekognition

AI23 hours ago

Arcanum makes Hungarian heritage accessible with Amazon Rekognition

AI23 hours ago

Arcanum makes Hungarian heritage accessible with Amazon Rekognition

AI23 hours ago

Arcanum makes Hungarian heritage accessible with Amazon Rekognition

AI23 hours ago

Arcanum makes Hungarian heritage accessible with Amazon Rekognition

AI23 hours ago

Arcanum makes Hungarian heritage accessible with Amazon Rekognition

AI23 hours ago

Arcanum makes Hungarian heritage accessible with Amazon Rekognition

AI23 hours ago

Arcanum makes Hungarian heritage accessible with Amazon Rekognition

AI23 hours ago

Arcanum makes Hungarian heritage accessible with Amazon Rekognition

AI23 hours ago

Arcanum makes Hungarian heritage accessible with Amazon Rekognition

AI23 hours ago

Arcanum makes Hungarian heritage accessible with Amazon Rekognition

AI23 hours ago

Arcanum makes Hungarian heritage accessible with Amazon Rekognition

AI23 hours ago

Arcanum makes Hungarian heritage accessible with Amazon Rekognition

Trending