About

MLlib is Spark's machine learning library, focusing on learning algorithms and utilities, including classification, regression, clustering, collaborative filtering, dimensionality reduction, as well as underlying optimization primitives.

We are launching a product which will configure and publish Spark MLlib, an open source software solution which is embedded pre-configured tool with Ubuntu OS and ready-to-launch VM on GCP that contains Spark MBlib, Hadoop 2.7, Scala, Linux, PHP (LAMP).

MLlib fits into Spark's APIs and interoperates with Scala. You can use any Hadoop data source (e.g. HDFS, HBase, or local files), making it easy to plug into Hadoop workflows.

Why MLlib? It is built on Apache Spark, which is a fast and general engine for large scale processing. Supposedly, running times or up to 100x faster than Hadoop MapReduce, or 10x faster on disk. Supports writing applications in Java, Scala, or Python.

MLlib contains many algorithms and utilities

Classification: logistic regression, naive Bayes
Regression: generalized linear regression, survival regression
Decision trees, random forests, and gradient-boosted trees
Recommendation: alternating least squares (ALS)
Clustering: K-means, Gaussian mixtures (GMMs)
Topic modeling: latent Dirichlet allocation (LDA)
Frequent itemsets, association rules, and sequential pattern mining
MLlib will still support the RDD-based API in spark.mllib with bug fixes.
MLlib will not add new features to the RDD-based API.
In the Spark 2.x releases, MLlib will add features to the DataFrames-based API to reach feature parity with the RDD-based API.
After reaching feature parity (roughly estimated for Spark 2.2), the RDD-based API will be deprecated.
The RDD-based API is expected to be removed in Spark 3.0.
DataFrames provide a more user-friendly API than RDDs. The many benefits of DataFrames include Spark Datasources, SQL/DataFrame queries, Tungsten and Catalyst optimizations, and uniform APIs across languages.
The DataFrame-based API for MLlib provides a uniform API across ML algorithms and across multiple languages.
DataFrames facilitate practical ML Pipelines, particularly feature transformations. See the Pipelines guide for details.
Data types
Classification and regression
Collaborative filtering
Clustering
Dimensionality reduction
Feature extraction and transformation

Usage/Deployment Instructions

1. Access the SparkMLlib in Google Marketplace and click the LAUNCH ON COMPUTE ENGINE button.

2. In the Configure & Deploy window, enter or select appropriate values for zone, machine type, and so on. Click the Deploy button.

3. The Google Developers Console confirms that SparkMLlib was deployed

To access SparkMLlib on external ip edit your firewall rule and open port 4040.

Now to Start SparkMLlib,

Step 1: Open 8080 Ports in the instance.

Step 2: Do SSH

$ sudo su

$ nano /etc/nginx/sites-available/spark.conf

Edit the configuration file and add your Public IP at the place of highlighted IP.

Save and close the file when you are finished.

$ systemctl restart nginx

$ cd /

$ cd /opt/spark

Run command to start

$ start-master.sh

Run command to stop

$ stop-master.sh

$ss-tunelp | grep 8080

Run command to start

$start-slave.sh

Run command to stop

$ stop-slave.sh

$ spark-shell

Step 2: Now Use the browser to access the application at http://<instance ip address> replace <instance ip address> with the actual IP address of the running instance.

All your queries are important to us. Please feel free to connect.

24X7 support provided for all the customers.

We are happy to help you.

Submit your Query: http://www.miritech.com/ContactUs.aspx

Contact Numbers:

(510) 298-5936
(510) 870-8668

Contact E-mail:

support@miritech.com

Submit Your Request

Name*

Email Address*

Phone Number

Message

Talk to Cloud Expert

You can subscribe to this product from GCP Marketplace and launch an instance from the product's image using the GCP compute service.

Go to the VM instances page.
Select your project and click Continue.
Click the Create instance button.
Specify a Name for your instance.
Optionally, change the Zone for this instance.

Note: The list of zones is randomized within each region to encourage use across multiple zones.

Select a Machine configuration for your instance.
In the Boot disk section, click Change to configure your boot disk.

Create a boot disk no larger than 2 TB to account for the limitations of MBR partitions.

Select the Custom Images tab.
Select the image project from the dropdown.
Choose the image you want and click the Select button.
To permit HTTP or HTTPS traffic to the VM instance, select Allow HTTP traffic or Allow HTTPS traffic.

The GCP Console adds a network tag to your instance and creates the corresponding ingress firewall rule that allows all incoming traffic on tcp:80 (HTTP) or tcp:443 (HTTPS). The network tag associates the firewall rule with the instance. For more information, see Firewall Rules Overview in the Virtual Private Cloud documentation.

To add secondary non-boot disks to your VM instance:
1. Click Management, security, disks, networking, sole tenancy.
2. Select the Disks tab.
3. Under Additional disks click Add new disk.
4. Specify a disk Name, Type, Source type, Mode, and Deletion rule.
5. Click Done.
6. Add additional disks as needed.
Click the Create button to create and start the instance.

1What is Hadoop?

The Apache Hadoop software library allows for the distributed processing of large data sets across clusters of computers using a simple programming model. The software library is designed to scale from single servers to thousands of machines; each server using local computation and storage. Instead of relying on hardware to deliver high-availability, the library itself handles failures at the application layer. As a result, the impact of failures is minimized by delivering a highly-available service on top of a cluster of computers.
2Where does Hadoop find applicability in business?

Hadoop, as a scalable system for parallel data processing, is useful for analyzing large data sets. Examples are search algorithms, market risk analysis, data mining on online retail data, and analytics on user behavior data.
3What is big data security analytics?

Add the words “information security” (or “cybersecurity” if you like) before the term “data sets” in the definition above. Security and IT operations tools spit out an avalanche of data like logs, events, packets, flow data, asset data, configuration data, and assortment of other things on a daily basis. Security professionals need to be able to access and analyze this data in real-time in order to mitigate risk, detect incidents, and respond to breaches. These tasks have come to the point where they are “difficult to process using on-hand data management tools or traditional (security) data processing applications.”
4Is there an easy way to migrate data from Hadoop into a relational database?

The Hadoop JDBC driver can be used to pull data out of Hadoop and then use the DataDirect JDBC Driver to bulk load the data into Oracle, DB2, SQL Server, Sybase, and other relational databases.
5Are Intelligent Assistants the only application of AI for customer care, or are there other ways that AI technologies can impact the contact center?

Front-end use of AI technologies to enable Intelligent Assistants for customer care is certainly key, but there are many other applications. One that I think is particularly interesting is the application of AI to directly support — rather than replace — contact center agents. Technologies such as natural language understanding and speech recognition can be used live during a customer service interaction with a human agent to look up relevant information and make suggestions about how to respond. AI technologies also have an important role in analytics. They can be used to provide an overview of activities within a call center, in addition to providing valuable business insights from customer activity.
6What are the popular machine learning algorithms in use today?
There are many machine learning algorithms in use today, but the most popular ones are:
- Decision Trees
- Naive Bayes Classification
- Ordinary Least Squares Regression
- Logistic Regression
- Support vector machines
- Ensemble Methods
- Clustering Algorithms
- Principal Component Analysis
- Singular Value Decomposition
- Independent Component Analysis

SparkMllib

About

Submit Your Request

1What is Hadoop?

2Where does Hadoop find applicability in business?

3What is big data security analytics?

4Is there an easy way to migrate data from Hadoop into a relational database?

5Are Intelligent Assistants the only application of AI for customer care, or are there other ways that AI technologies can impact the contact center?

6What are the popular machine learning algorithms in use today?

Highlights

Application Installed