Azure Marketplace Prerequisites

To automatically deploy and configure the Azure Datalake that will be used by Indexima for data storage, you need to setup an Azure managed-identity.

For most of the requirements listed below, you can use the Azure portal way or the PowerShell commands way.

If you want use the PowerShell way, you must install it. For that, go to this URL: https://github.com/PowerShell/PowerShell/releases and select the latest stable release of PowerShell (7.0.3 at the time of writing of this documentation).

In the assets linked with your selected release, choose the package corresponding to your OS and download it. After that, proceed with the installation, depending of your OS.

All of this documentation assumes that you have already created the target resource group. In the context of this documentation, this resource group is named “test-rg”.

The user you select to connect to Azure must have the following properties:

Must be a member of “Global administrator” and “Application administrator” roles
Must have a “Owner” RBAC on the Azure subscription

Azure portal

Make sure the “Microsoft.ContainerInstance” provider is registered

Your subscription must have the “Microsoft.ContainerInstance” provider as registered because the deployment scripts resource in ARM templates need a Container Instance and a Storage Account to work.

Follow these steps for that:

Access to Azure portal with a global administrator privilege user account (https://portal.azure.com).
Deploy burger menu and click on “All services” blade:
Click en “All services”:
Click en “Subscriptions”:
Select your subscription:
Select “Resource providers” from “Settings” section:
In the right main panel, make a search with "containerinstance":
Check if the found provider "Microsoft.ContainerInstance" is registered and if not, click on the "Register" button:
Check the status of registration by clicking on “Refresh” button:

User-assigned managed identity

Deployment of scripts resource in ARM templates requires a user-assigned managed identity.

The documentation on this subject can be found here: https://docs.microsoft.com/en-us/azure/azure-resource-manager/templates/deployment-script-template?tabs=CLI#sample-templates.

Create user-assigned managed identity

To create a user-assigned managed identity, follow these steps:

Access to the Azure portal using a global administrator privilege user account (https://portal.azure.com).
Search for "Managed Identities":
Click on "+ Add":
Name the user-assigned managed identity "indexima" (for example), select your target resource group and a location (“West Europe” is possible):
Click on “Create” at the bottom to confirm the new user-assigned managed identity:

Make your user-assigned managed identity as “Owner” on the target resource group

Access the Azure portal using a global administrator privilege user account (https://portal.azure.com).
Deploy the burger menu and select “Resource groups”:
Select your target resource group from the list of your resource groups (“test-rg” for example):
Select "Access Control (IAM)":
Select the "Role assignments" tab:
Click on "+ Add":
Select "Add role assignment":
In the right panel, select "Owner" on the Role dropdown:
Search for "indexima" (your newly created user-assigned managed identity):
Don't forget to click on "Save" at the bottom:

Add your user-assigned managed identity as a member of the “Application administrator” role

Access the Azure portal using a global administrator privilege user account (https://portal.azure.com).
Deploy the burger menu and select “Azure Active Directory”:
Click on "Roles and administrators" from the “Manage” section:
In the right list, click on "Application administrator":
Click on "+ Add assignments":
In the right panel, search for your newly created user-assigned managed identity "indexima" and select it:
Don’t forget to click on "Add" at the bottom:

Use the same steps to make the user-assigned managed identity as member of “Global administrator” role (only select “Global administrator” role in place of “Application administrator”).

PowerShell

As a PowerShell Core script, you can use it on both Windows and Linux OS.

Important: This PowerShell script will ask for a login to your Azure subscription. Make sure to login to a user account with “Global administrator” and “Application administrator” privileges. You will also need "Owner" privileges on your Azure subscription. Otherwise, this script will fail.

POWERSHELL

#Requires -Version 7

<#
.SYNOPSIS
Install and configure all dependencies for Indexima Azure Application deployment.

.DESCRIPTION
The Azure Application offer of Indexima must have a user-assigned managed identity
and some others dependencies like modules for PowerShell.
This PowerShell Core script create all necessary items to allow correct execution
of the Azure Application deployment. It must be executed before the request of 
provisionning of the Azure Application.

.PARAMETER ResourceGroupName
Specify the existing resource group name where the managed identity will be created. You can use 'rg' alias.

.PARAMETER TargetResourceGroupName
Specify the target resource group name where the resources will be deployed. You can use 'target' alias.

.PARAMETER Location
Specify the location of the managed identity to create. You can use 'loc' alias.

.PARAMETER ManagedIdentityName
Specify managed identity name to create. You can use 'mi' alias.

.EXAMPLE
.\Install-Requirements.ps1 -ResourceGroupName test-rg -TargetResourceGroupName target-rg -Location westeurope -ManagedIdentityName indexima-mi

.EXAMPLE
.\Install-Requirements.ps1 -rg test-rg -target target-rg -loc westeurope -mi indexima-mi

.LINK
https://indexima.com

.NOTES
Version: 1.1.0
Author: Indexima <support@indexima.com>
Ownership: Indexima © 2020 - All rights reserved
#>

# Define parameters
[CmdletBinding()]
Param
(
    [Parameter(Mandatory = $true)]
    [Alias("rg")]
    [String]$ResourceGroupName,
    [Parameter(Mandatory = $true)]
    [Alias("target")]
    [String]$TargetResourceGroupName,
    [Parameter(Mandatory = $true)]
    [ValidateSet("westeurope", "northeurope", "eastus2", "centralus", "australiaeast")]
    [Alias("loc")]
    [String]$Location,
    [Parameter(Mandatory = $true)]
    [Alias("mi")]
    [String]$ManagedIdentityName
)

# Define some variables
$aad_app_name_guid = New-Guid
$aad_app_name = ("indexima{0}" -f $aad_app_name_guid)
$aad_app_password_guid = New-Guid
$aad_app_password_guid_bytes = [System.Text.Encoding]::Unicode.GetBytes($aad_app_password_guid)
$aad_app_password = [Convert]::ToBase64String($aad_app_password_guid_bytes).Substring(0, 16)
$aad_app_secure_password = ConvertTo-SecureString -String $aad_app_password -AsPlainText -Force

# Start
Write-Host "Start to install dependencies for your Indexima Azure Application..."
Write-Host "===================================================================="

# Install mandatory modules (Az.Resources, Az.Accounts)
$has_az_resources_module_installed = Get-InstalledModule | Where-Object { $_.Name -eq "Az.Resources" }
if ($null -eq $has_az_resources_module_installed) {
    Write-Host "Install Azure Resources module"
    Install-Module -Name Az.Resources -AllowClobber -Scope CurrentUser -Force
}
$has_az_accounts_module_installed = Get-InstalledModule | Where-Object { $_.Name -eq "Az.Accounts" }
if ($null -eq $has_az_accounts_module_installed) {
    Write-Host "Install Azure Resources module"
    Install-Module -Name Az.Accounts -AllowClobber -Scope CurrentUser -Force
}
$has_az_msi_module_installed = Get-InstalledModule | Where-Object { $_.Name -eq "Az.ManagedServiceIdentity" }
if ($null -eq $has_az_msi_module_installed) {
    Write-Host "Install Azure Managed Service Identity module"
    Install-Module -Name Az.ManagedServiceIdentity -Scope CurrentUser -AllowPrerelease -Force
    Import-Module Az.ManagedServiceIdentity -Force
}

# Make a logout from the current Azure account
Write-Host "Make a logout from the current Azure account"
$null = Disconnect-AzAccount

# Request for a login to a Azure account
Write-Host "Connect to Azure account"
$null = Connect-AzAccount

# Install Container Instance provider for subscription if necessary
$has_subscription_provider_installed = Get-AzResourceProvider -ProviderNamespace Microsoft.ContainerInstance | Where-Object { $_.RegistrationState -eq "Registered" } | Select-Object -Index 0
if ($null -eq $has_subscription_provider_installed) {
    Write-Host "Install Container Instance provider for subscription"
    $null = Register-AzResourceProvider -ProviderNamespace Microsoft.ContainerInstance
}

# Create target resource group if not exist
Write-Host "Create target resource group if not exist"
$target_resource_group_not_exists = $true
$null = Get-AzResourceGroup -Name $TargetResourceGroupName -ErrorVariable target_resource_group_not_exists -ErrorAction SilentlyContinue
if ($target_resource_group_not_exists) {
    $null = New-AzResourceGroup -Name $TargetResourceGroupName -Location $Location
}

# Create managed identity
Write-Host "Create managed identity"
$null = New-AzUserAssignedIdentity -ResourceGroupName $ResourceGroupName -Name $ManagedIdentityName -Location $Location

# Get current subscription id
Write-Host "Get current subscription id"
$tenant_id = (Get-AzContext).Tenant.Id
$subscription_id = (Get-AzSubscription -TenantId $tenantId).Id

# Wait for 10 seconds to make sure managed identity has been provisioned
Start-Sleep -Seconds 10

# Apply owner RBAC to target resource group
Write-Host "Apply owner RBAC to target resource group"
$managed_identity_object_id = (Get-AzUserAssignedIdentity -Name $ManagedIdentityName -ResourceGroupName $ResourceGroupName).PrincipalId
$null = New-AzRoleAssignment -ObjectId $managed_identity_object_id -RoleDefinitionName Owner -Scope /subscriptions/$subscription_id/resourceGroups/$TargetResourceGroupName

# Apply contributor RBAC to subscription
Write-Host "Apply contributor RBAC to subscription"
$null = New-AzRoleAssignment -ObjectId $managed_identity_object_id -RoleDefinitionName Contributor -Scope /subscriptions/$subscription_id

# Create AAD application
Write-Host "Create AAD application"
$aad_app = New-AzADApplication -DisplayName $aad_app_name -Password $aad_app_secure_password -IdentifierUris "http://$aad_app_name"

# Wait for the user says 'yes' after apply of API permissions to created AAD application
$prompt_title = "Using the Azure portal, please add 'RoleManagement.ReadWrite.Directory' and 'Directory.Read.All' permissions to '{0}' AAD application" -f $aad_app_name
$prompt_label = "Have you added these permissions?"
$prompt_validate_option = New-Object System.Management.Automation.Host.ChoiceDescription "&Yes", "Yes, I added the permissions"
$prompt_cancel_option = New-Object System.Management.Automation.Host.ChoiceDescription "&No", "No, I want to interrupt this installation"
$prompt_options = [System.Management.Automation.Host.ChoiceDescription[]] ($prompt_validate_option, $prompt_cancel_option) 
$prompt_user_input = $Host.UI.PromptForChoice($prompt_title, $prompt_label, $prompt_options, 0)
if ($prompt_user_input -ne 0) {
    Write-Host "Installation interrupted"
    exit 1
}

# Make a request to get access token
Write-Host "Get access token"
$tenant_id = (Get-AzTenant).Id
$token_uri = ("https://login.microsoftonline.com/{0}/oauth2/v2.0/token" -f $tenant_id)
$token_body = @{
    client_id     = $aad_app.ApplicationId
    scope         = "https://graph.microsoft.com/.default"
    client_secret = $aad_app_password
    grant_type    = "client_credentials"
}
$token_request = Invoke-WebRequest -Method "POST" -Uri $token_uri -ContentType "application/x-www-form-urlencoded" -Body $token_body -UseBasicParsing
$access_token = ($token_request.Content | ConvertFrom-Json).access_token

# Check if Global (or Company) Administrator and Application Administrator roles are activated
Write-Host "Check if Global Administrator and Application Administrator roles are activated"
$directory_roles_uri = "https://graph.microsoft.com/v1.0/directoryRoles"
$directory_roles_request = Invoke-WebRequest -Method "GET" -Uri $directory_roles_uri -ContentType "application/json" -Headers @{ Authorization = "Bearer $access_token" }
$directory_roles_list = ($directory_roles_request.Content | ConvertFrom-Json).value
$has_app_admin_role = ($directory_roles_list | Where-Object { $_.DisplayName -eq "Application Administrator" }).Id
$has_global_admin_role = ($directory_roles_list | Where-Object { $_.DisplayName -eq "Company Administrator" }).Id

# Get role template id for Global Administrator and Application Administrator
if (($null -eq $has_app_admin_role) -or ($null -eq $has_global_admin_role)) {
    Write-Host "Get role template id for Global Administrator and Application Administrator"
    $directory_role_templates_uri = "https://graph.microsoft.com/v1.0/directoryRoleTemplates"
    $directory_role_templates_request = Invoke-WebRequest -Method "GET" -Uri $directory_role_templates_uri -ContentType "application/json" -Headers @{ Authorization = "Bearer $access_token" }
    $directory_role_templates_list = ($directory_role_templates_request.Content | ConvertFrom-Json).value
    if ($null -eq $has_app_admin_role) {
        $app_admin_template_role_id = ($directory_role_templates_list | Where-Object { $_.DisplayName -eq "Application Administrator" }).Id
    }
    if ($null -eq $has_global_admin_role) {
        $global_admin_template_role_id = ($directory_role_templates_list | Where-Object { $_.DisplayName -eq "Company Administrator" }).Id
    }
}

# Activate Application Administrator role if not already activated
$directory_roles_uri = "https://graph.microsoft.com/v1.0/directoryRoles"
if ($null -eq $has_app_admin_role) {
    Write-Host "Activate Application Administrator role if not already activated"
    $activate_app_admin_role_body = @{
        roleTemplateId = $app_admin_template_role_id
    }
    $activate_app_admin_role_request = Invoke-WebRequest -Method "POST" -Uri $directory_roles_uri -ContentType "application/json" -Headers @{ Authorization = "Bearer $access_token" } -Body ($activate_app_admin_role_body | ConvertTo-Json)
    $has_app_admin_role = ($activate_app_admin_role_request.Content | ConvertFrom-Json).Id
}

# Activate Global Administrator role if not already activated
if ($null -eq $has_global_admin_role) {
    Write-Host "Activate Global Administrator role if not already activated"
    $activate_global_admin_role_body = @{
        roleTemplateId = $global_admin_template_role_id
    }
    $activate_global_admin_role_request = Invoke-WebRequest -Method "POST" -Uri $directory_roles_uri -ContentType "application/json" -Headers @{ Authorization = "Bearer $access_token" } -Body ($activate_global_admin_role_body | ConvertTo-Json)
    $has_global_admin_role = ($activate_global_admin_role_request.Content | ConvertFrom-Json).Id
}

# Add managed identity as member of Application Administrator directory role
Write-Host "Add managed identity as member of Application Administrator directory role"
$directory_roles_app_admin_member_uri = "https://graph.microsoft.com/v1.0/directoryRoles/$has_app_admin_role/members/`$ref"
$directory_roles_member_body = @{
    "@odata.id" = "https://graph.microsoft.com/v1.0/directoryObjects/$managed_identity_object_id"
}
$null = Invoke-WebRequest -Method "POST" -Uri $directory_roles_app_admin_member_uri -ContentType "application/json" -Headers @{ Authorization = "Bearer $access_token" } -Body ($directory_roles_member_body | ConvertTo-Json)

# Add managed identity as member of Global Administrator directory role
Write-Host "Add managed identity as member of Global Administrator directory role"
$directory_roles_global_admin_member_uri = "https://graph.microsoft.com/v1.0/directoryRoles/$has_global_admin_role/members/`$ref"
$null = Invoke-WebRequest -Method "POST" -Uri $directory_roles_global_admin_member_uri -ContentType "application/json" -Headers @{ Authorization = "Bearer $access_token" } -Body ($directory_roles_member_body | ConvertTo-Json)

# Remove AAD application
Write-Host "Remove AAD application"
$null = Remove-AzADApplication -ApplicationId $aad_app.ApplicationId -Force

# Say is ended
Write-Host "===================================================================="
Write-Host "Dependencies correctly installed for your Indexima Azure Application. Wait some two or three minutes to make sure all necessary ressources have been deployed and proceed to request of provisioning of your Indexima Azure Application."