Architecture Diagram

Elements created by the Indexima AWS Marketplace CloudFormation template :

VPC

The default template does not create the VPC resources. It is assumed you already have a VPC ID and a Subnet ID to deploy the application to.

Unless your Dataviz servers are on the same VPC, it will be required that the target Subnet is a public subnet.

Security groups

Two security groups are created by the template.

UI Security group

The UI EC2 will need to open the ports on the allowed CIDR for

SSH (22)
Indexima Developer Console Web application (8082)
Zookeeper client connection (2181)

All traffic is open on the UI and Indexima security groups source.

Indexima Security group

The Indexima EC2 machines need to open the ports on the allowed CIDR for

SSH (22)
Indexima Monitor API (9999)
Indexima Hive server (10000)

All traffic is open on the UI and Indexima security groups source.

IAM

For the IAM resources, the template creates a role that can be assumed by an EC2, a custom policy, and an instance profile.

Indexima Role

The role needs to allow EC2 resources to assume it.

Policy

The policy attached to the role allows to list all the bucket in the account, and allows all operations on the Indexima bucket and prefix (these values are inputted as parameters of the template).

Instance Profile

In order to attach a role to an EC2 instance, an instance profile needs to be created. The role above is attached to it.

Compute

A number of on-demand EC2 instances will be created: an Indexima master machine, the UI machine, and the Zookeeper server.

An autoscaling group and a launch configuration will also be created for the Indexima workers.

Resource Type	Resource Name	Description
On-demand EC2	Indexima Master	The primary machine Indexima needs to start.
On-demand EC2	Indexima UI	The machine hosting the web application allowing the administration of the Indexima cluster.
On-demand EC2	Zookeeper	A server with Zookeeper is installed on this machine. All Indexima instances join the Zookeeper namespace, allowing them to be reachable thru Zookeeper.
Autoscaling Group	Indexima Cluster ASG	The worker instances are created with an autoscaling group. Its initial size is the size inputted in the Cluster Size parameter. It can be changed by updating the CloudFormation stack and changing this value.
Launch Config	Indexima Cluster LC	The configuration and start script for the Indexima workers. Similar to the Indexima Master configuration except for the fact that it needs the master to be up and running so the workers can attach themselves to the Master.