Trying to setup diagnostic settings for activity logs in ARM template - c#

I've been trying to create a template for a logger function app in Azure, the basic idea is that it'll forward all activity log category info along with all audit log info using event hubs as triggers, I've gotten down most of the template, but I keep hitting the following error when it comes to the diagnostic settings part:
The resource type '/' does not support diagnostic settings.
Is there something I'm doing wrong? I've only started working with azure for a short while, and am afraid I'm misunderstanding how diagnostic settings are created in an ARM template. I can't seem to find any template or documentation online about this, and am at a loss...
Here's an excerpt:
...
"variables": {
"eventHubNamespaceName": "azure-logger",
"auditEventHubName": "[concat(variables('eventHubNamespaceName'), '/', 'audit-logs')]",
"activityEventHubName": "[concat(variables('eventHubNamespaceName'), '/', 'activity-logs')]",
"authRuleResourceId": "[resourceId('Microsoft.Eventhub/namespaces/authorizationRules', variables('eventHubNamespaceName'), 'RootManageSharedAccessKey')]",
"auditAuthorizationRuleId": "[resourceId('Microsoft.EventHub/namespaces/eventhubs/authorizationRules', variables('eventHubNamespaceName') , 'audit-logs', 'default')]",
"activityAuthorizationRuleId": "[resourceId('Microsoft.EventHub/namespaces/eventhubs/authorizationRules', variables('eventHubNamespaceName') , 'activity-logs', 'default')]"
},
"resources": [
{
"name": "[variables('eventHubNamespaceName')]",
"type": "Microsoft.EventHub/namespaces",
"apiVersion": "2021-06-01-preview",
"location": "[parameters('location')]",
"sku": {
"name": "Standard"
},
"identity": {
"type": "SystemAssigned"
},
"properties": {
"status": "Active"
},
"resources": [
{
"type": "Microsoft.EventHub/namespaces/eventhubs",
"apiVersion": "2021-06-01-preview",
"name": "[concat(variables('eventHubNamespaceName'), '/', 'audit-logs')]",
"location": "[parameters('location')]",
"dependsOn": [
"[resourceId('Microsoft.EventHub/namespaces', variables('eventHubNamespaceName'))]"
],
"properties": {
"messageRetentionInDays": 1,
"partitionCount": 1
},
"resources": [
{
"type": "Microsoft.EventHub/namespaces/eventhubs/authorizationRules",
"apiVersion": "2021-06-01-preview",
"name": "[concat(variables('eventHubNamespaceName'), '/', 'audit-logs','/','default')]",
"properties": {
"rights": [ "Manage", "Listen", "Send" ]
},
"dependsOn": [
"[resourceId('Microsoft.EventHub/namespaces/eventhubs', variables('eventHubNamespaceName'), 'audit-logs')]"
]
}
]
},
{
"type": "Microsoft.EventHub/namespaces/eventhubs",
"apiVersion": "2021-06-01-preview",
"name": "[concat(variables('eventHubNamespaceName'), '/', 'activity-logs')]",
"location": "[parameters('location')]",
"dependsOn": [
"[resourceId('Microsoft.EventHub/namespaces', variables('eventHubNamespaceName'))]"
],
"properties": {
"messageRetentionInDays": 1,
"partitionCount": 1
},
"resources": [
{
"type": "Microsoft.EventHub/namespaces/eventhubs/authorizationRules",
"apiVersion": "2021-06-01-preview",
"name": "[concat(variables('eventHubNamespaceName'), '/', 'activity-logs','/','default')]",
"properties": {
"rights": [ "Manage", "Listen", "Send" ]
},
"dependsOn": [
"[resourceId('Microsoft.EventHub/namespaces/eventhubs', variables('eventHubNamespaceName'), 'activity-logs')]"
]
}
]
}
]
},
{
"type": "Microsoft.Insights/diagnosticSettings",
"apiVersion": "2017-05-01-preview",
"name": "[concat(variables('eventHubNamespaceName'),'activity-logs')]",
"properties": {
"eventHubName": "activity-logs",
"eventHubAuthorizationRuleId": "[variables('activityAuthorizationRuleId')]",
"logs": [
{
"category": "Administrative",
"enabled": true
},
{
"category": "Security",
"enabled": true
},
{
"category": "ServiceHealth",
"enabled": true
},
{
"category": "Alert",
"enabled": true
},
{
"category": "Recommendation",
"enabled": true
},
{
"category": "Policy",
"enabled": true
},
{
"category": "Autoscale",
"enabled": true
},
{
"category": "ResourceHealth",
"enabled": true
}
]
}
}
]
...```

Diagnostic settings for Activity logs are created for a subscription, not for a resource group like settings for Azure resources. Therefore, when you check the Diagnostic setting for Activity log in this document, you will find that you can deploy Azure Resource Manager templates using any valid method including PowerShell and CLI. Diagnostic settings for Activity log must deploy to a subscription using az deployment create for CLI or New-AzDeployment for PowerShell.
And the template should look like as following :
{
"$schema": "https://schema.management.azure.com/schemas/2018-05-01/subscriptionDeploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"settingName": {
"type": "String"
},
"workspaceId": {
"type": "String"
},
"storageAccountId": {
"type": "String"
},
"eventHubAuthorizationRuleId": {
"type": "String"
},
"eventHubName": {
"type": "String"
}
},
"resources": [
{
"type": "Microsoft.Insights/diagnosticSettings",
"apiVersion": "2017-05-01-preview",
"name": "[parameters('settingName')]",
"properties": {
"workspaceId": "[parameters('workspaceId')]",
"storageAccountId": "[parameters('storageAccountId')]",
"eventHubAuthorizationRuleId": "[parameters('eventHubAuthorizationRuleId')]",
"eventHubName": "[parameters('eventHubName')]",
"logs": [
{
"category": "Administrative",
"enabled": true
},
{
"category": "Security",
"enabled": true
},
{
"category": "ServiceHealth",
"enabled": true
},
{
"category": "Alert",
"enabled": true
},
{
"category": "Recommendation",
"enabled": true
},
{
"category": "Policy",
"enabled": true
},
{
"category": "Autoscale",
"enabled": true
},
{
"category": "ResourceHealth",
"enabled": true
}
]
}
}
]
}
You should have a Parameter file if you follow the above template.
Also note that when virtual networks are enabled Diagnostic Settings can't access Event Hubs resources. You have to enable the Allow trusted Microsoft services to bypass this firewall setting in Event Hub, so that Diagnostic Settings (Azure Monitor Services) is granted access to your Event Hubs resources.

Related

Google API Explorer doesn't create Compute Engine VM instance

as in title, I am trying to use Google API Explorer to run vm instance. I am using instances.insert for this, but I can't get it to work. After successfuly executing the call I can not see any newly creted vm instance in https://console.cloud.google.com/compute/instances
The request I am trying to execute is copied from Equivalent REST request in Google Cloud Console Create an instance web page :
{
"name": "some-name",
"machineType": "projects/my-project-id/zones/europe-west3-c/machineTypes/f1-micro",
"displayDevice": {
"enableDisplay": false
},
"metadata": {
"items": [
{
"key": "startup-script",
"value": "#! /bin/bash\necho hello\nEOF"
}
]
},
"tags": {
"items": []
},
"disks": [
{
"type": "PERSISTENT",
"boot": true,
"mode": "READ_WRITE",
"autoDelete": true,
"deviceName": "some-name",
"initializeParams": {
"sourceImage": "projects/debian-cloud/global/images/debian-10-buster-v20200910",
"diskType": "projects/my-project-id/zones/europe-west3-c/diskTypes/pd-standard",
"diskSizeGb": "10",
"labels": {}
},
"diskEncryptionKey": {}
}
],
"canIpForward": false,
"networkInterfaces": [
{
"subnetwork": "projects/my-project-id/regions/europe-west3/subnetworks/default",
"accessConfigs": [
{
"name": "External NAT",
"type": "ONE_TO_ONE_NAT",
"networkTier": "PREMIUM"
}
],
"aliasIpRanges": []
}
],
"description": "",
"labels": {},
"scheduling": {
"preemptible": false,
"onHostMaintenance": "MIGRATE",
"automaticRestart": true,
"nodeAffinities": []
},
"deletionProtection": false,
"reservationAffinity": {
"consumeReservationType": "ANY_RESERVATION"
},
"serviceAccounts": [
{
"email": "some-number-compute#developer.gserviceaccount.com",
"scopes": [
"https://www.googleapis.com/auth/cloud-platform"
]
}
],
"shieldedInstanceConfig": {
"enableSecureBoot": false,
"enableVtpm": true,
"enableIntegrityMonitoring": true
},
"confidentialInstanceConfig": {
"enableConfidentialCompute": false
}
}
Here is the response with status 200
{
"id": "2981010757915612255",
"name": "operation-1602235056020-5b1396b5c5cee-e0e30499-4d06ce75",
"zone": "https://www.googleapis.com/compute/v1/projects/my-project-id/zones/europe-west3-c",
"operationType": "insert",
"targetLink": "https://www.googleapis.com/compute/v1/projects/my-project-id/zones/europe-west3-c/instances/ams2-linux-race-1",
"targetId": "1541614827291382879",
"status": "RUNNING",
"user": "email#gmail.com",
"progress": 0,
"insertTime": "2020-10-09T02:17:36.818-07:00",
"startTime": "2020-10-09T02:17:36.821-07:00",
"selfLink": "https://www.googleapis.com/compute/v1/projects/my-project-id/zones/europe-west3-c/operations/operation-1602235056020-5b1396b5c5cee-e0e30499-4d06ce75",
"kind": "compute#operation"
}
I have the same issue with C# code example from https://cloud.google.com/compute/docs/reference/rest/v1/instances/insert#examples
I can execute the same request without errors and in response I am getting this
{
"clientOperationId":null,
"creationTimestamp":null,
"description":null,
"endTime":null,
"error":null,
"httpErrorMessage":null,
"httpErrorStatusCode":null,
"id":3283200477858999168,
"insertTime":"2020-10-09T00:46:55.187-07:00",
"kind":"compute#operation",
"name":"operation-1602229614262-5b1382701b989-381126a6-cc145485",
"operationType":"insert",
"progress":0,
"region":null,
"selfLink":"https://www.googleapis.com/compute/v1/projects/my-project-id/zones/europe-west3-c/operations/operation-1602229614262-5b1382701b989-381126a6-cc145485",
"startTime":"2020-10-09T00:46:55.189-07:00",
"status":"RUNNING",
"statusMessage":null,
"targetId":2365846324436118401,
"targetLink":"https://www.googleapis.com/compute/v1/projects/my-project-id/zones/europe-west3-c/instances/some-name",
"user":"email#gmail.com",
"warnings":null,
"zone":"https://www.googleapis.com/compute/v1/projects/my-project-id/zones/europe-west3-c",
"ETag":null
}
but I can't see any new instance beeing created...
Does any one know what is the issue here?
The Compute Engine API is enabled. Result of gcloud services list:
NAME TITLE
...
compute.googleapis.com Compute Engine API
...
Can you please double check if Compute Engine Api is enabled and post the result in your question.
gcloud services list
I believe your Compute Engine Api is not enabled.

Message card rendering issue in bot framework v3

I have a message card payload pushed from service now which is not rendering properly on MS Teams through bot framework. It only displays the content but not the button.Below is the payload, pls suggest what could be the issue.
BotFramework:V3
.Net SDK
Bot Builder package:3.12.2.4
Bot Connector:3.12.2.4
Bot.Connector.Teams:0.9.0
{
"contentType": "application/vnd.microsoft.teams.card.o365connector",
"content": {
"#type": "MessageCard",
"#context": "http://schema.org/extensions",
"title": "Incident Updated - INC0010010",
"summary": "Incident Updated - INC0010010",
"themeColor": "D1222B",
"sections": [
{
"title": "",
"text": "",
"activityTitle": "",
"activitySubtitle": "",
"activityText": "",
"facts": [
{
"name": "Category",
"value": "Software"
},
{
"name": "State",
"value": "New"
},
{
"name": "Priority",
"value": "5 - Planning"
},
{
"name": "Assignment group",
"value": "Software"
}
]
},
{
"text": "Please check the details"
}
],
"potentialAction": [
{
"#type": "OpenUri",
"name": "View Incident in ServiceNow",
"targets": [
{
"os": "default",
"uri": "https://dev62584.service-now.com/incident.do?sys_id=XXXXXXX&sysparm_stack=incident_list.do?sysparm_query=active=true"
},
{
"os": "iOS",
"uri": "https://dev62584.service-now.com/incident.do?sys_id=XXXXXXX&sysparm_stack=incident_list.do?sysparm_query=active=true"
},
{
"os": "android",
"uri": "https://dev62584.service-now.com/incident.do?sys_id=XXXXXXX&sysparm_stack=incident_list.do?sysparm_query=active=true"
},
{
"os": "windows",
"uri": "https://dev62584.service-now.com/incident.do?sys_id=7ec9865adb711010fcff8809489619b4&sysparm_stack=incident_list.do?sysparm_query=active=true"
}
]
}
]
}
}
Microsoft Troubleshooting and recommendation:
We were finally able to repro this issue. It is a bug in the team's scope and doesn't repro in the personal scope. An easy way to unblock you would be to add in the potentialAction section an #id field like so:
"potentialAction": [
{
"#id":"1",
"#type": "OpenUri",
Nothing stands out as a bug in your code or problem with your configuration. Linking to the GitHub issue where it is being investigated/troubleshot:
https://github.com/microsoft/BotBuilder-V3/issues/208

Create DropDown list for jira custom field in REST API

I'm trying to retrieve the available options for fields through the Jira REST API but I'm getting stuck on the custom fields.
A normal field returns something like this:
{[priority, {
"required": false,
"schema": {
"type": "priority",
"system": "priority"
},
"name": "Priority",
"key": "priority",
"hasDefaultValue": true,
"operations": [
"set"
],
"allowedValues": [
{
"self": "https://inn-inw.atlassian.net/rest/api/2/priority/1",
"iconUrl": "https://inn-inw.atlassian.net/images/icons/priorities/highest.svg",
"name": "Highest",
"id": "1"
},
{
"self": "https://inn-inw.atlassian.net/rest/api/2/priority/2",
"iconUrl": "https://inn-inw.atlassian.net/images/icons/priorities/high.svg",
"name": "High",
"id": "2"
},
{
"self": "https://inn-inw.atlassian.net/rest/api/2/priority/3",
"iconUrl": "https://inn-inw.atlassian.net/images/icons/priorities/medium.svg",
"name": "Medium",
"id": "3"
},
{
"self": "https://inn-inw.atlassian.net/rest/api/2/priority/4",
"iconUrl": "https://inn-inw.atlassian.net/images/icons/priorities/low.svg",
"name": "Low",
"id": "4"
},
{
"self": "https://inn-inw.atlassian.net/rest/api/2/priority/5",
"iconUrl": "https://inn-inw.atlassian.net/images/icons/priorities/lowest.svg",
"name": "Lowest",
"id": "5"
}
],
"defaultValue": {
"self": "https://inn-inw.atlassian.net/rest/api/2/priority/3",
"iconUrl": "https://inn-inw.atlassian.net/images/icons/priorities/medium.svg",
"name": "Medium",
"id": "3"
}
}]}
The custom fields on the other hand return a much "smaller" set!
"Sprint" Custom Field:
{[customfield_10113, {
"required": false,
"schema": {
"type": "array",
"items": "string",
"custom": "com.pyxis.greenhopper.jira:gh-sprint",
"customId": 10113
},
"name": "Sprint",
"key": "customfield_10113",
"hasDefaultValue": false,
"operations": [
"set"
]
}]}
"Epic Link"
{[customfield_10006, {
"required": false,
"schema": {
"type": "any",
"custom": "com.pyxis.greenhopper.jira:gh-epic-link",
"customId": 10006
},
"name": "Epic Link",
"key": "customfield_10006",
"hasDefaultValue": false,
"operations": [
"set"
]
}]}
As you can see there are no options available, but i know that there are.
How can I get to these values through rest?
I have tried checking the request that the cloud server makes but it's pretty different for each of the custom fields and i cannot find a relation between them. For example:
To get the sprint possible values: https://inn-inw.atlassian.net/rest/greenhopper/1.0/sprint/picker?query=
and to get the Epic values: https://inn-inw.atlassian.net/rest/greenhopper/1.0/epics
I'm trying to make things as clean as possible and I would prefer not to hard code the request, specially because i would probably need to update the code every time a new custom field is created.
PS: I have tried some of the solutions presented in other questions but couldn't find one that fits what i need. eg: https://inn-inw.atlassian.net//rest/api/2/issue/createmeta?projectKeys=PROJKEY&issuetypeNames=Bug&expand=projects.issuetypes.fields returns only the short description that is displayed above.

Timeout when indexing document with custom analyzer

I am creating the mappings for an index I will be using in a project.
Given the domain of the features, I'd like most of the fields to be searchable through case-insensitive term queries.
I had worked through a custom analyzer (like the one suggested here: Elasticsearch Map case insensitive to not_analyzed documents) but when I try to index a document, the process hangs for 60 seconds until a timeout happens and the whole process fails.
I see the same behavior when I test on Sense.
Here is the index definition:
put /emails
{
"mappings": {
"email": {
"properties": {
"createdOn": {
"type": "date",
"store": true,
"format": "strict_date_optional_time||epoch_millis"
},
"data": {
"type": "object",
"dynamic": "true"
},
"from": {
"type": "string",
"store": true
},
"id": {
"type": "string",
"store": true
},
"sentOn": {
"type": "date",
"store": true,
"format": "strict_date_optional_time||epoch_millis"
},
"sesId": {
"type": "string",
"store": true
},
"subject": {
"type": "string",
"store": true,
"analyzer": "standard"
},
"templates": {
"properties": {
"html": {
"type": "string",
"store": true
},
"plainText": {
"type": "string",
"store": true
}
}
},
"to": {
"type": "string",
"store": true
},
"type": {
"type": "string",
"store": true
}
}
},
"event": {
"_parent": {
"type": "email"
},
"properties": {
"id": {
"type": "string",
"store": true
},
"origin": {
"type": "string",
"store": true
},
"time": {
"type": "date",
"store": true,
"format": "strict_date_optional_time||epoch_millis"
},
"type": {
"type": "string",
"store": true
},
"userAgent": {
"type": "string",
"store": true
}
}
}
},
"settings": {
"number_of_shards": "5",
"number_of_replicas": "0",
"analysis": {
"analyzer": {
"default": {
"tokenizer": "keyword",
"filter": [
"lowercase"
],
"type": "custom"
}
}
}
}
}
As you can see, I define an analyzer as "default" (if I try to use another name and define it as a default analyzer for each of the two types, I get a "Root mapping definition has unsupported parameters: [analyzer : my_analyzer]" error).
And this is me trying to add a document to the index
post /emails/email/1
{
"from": "email-address-1",
"to": "email-address-2",
"subject": "Hello world",
"data":{
"status": "SENT"
}
}
I really can't understand why this timeout is happening.
I also tried using NEST via a C# console application. Same behavior.
Thanks.
PS: for testing I am using both Elasticsearch 2.3 hosted by AWS and Elasticsearch 2.3 hosted in a local docker container.
The problem is that you have 1 node and an index with 1 primary shard and 5 replica shards.
Since replicas of a primary will not be assigned on the same node as the primary, the 5 replicas will all be unassigned. This is an issue when indexing a document; by default, the write consistency for an index operation is quorum, and a quorum of 6 (1 primary + 5 replicas) is 4 (n/2 + 1). This means the document needs to have been written to the primary and 3 replicas of the same shard in order to succeed. With unassigned shards, it won't be possible to satisfy this. You'll see a UnavailableShardsException in the logs with an error message for this.
Changing your index to 5 shards and 1 replica will solve the problem.

Deploy a .NET Windows Service with Amazon Elastic Beanstalk with no Web Application

I want to create an Elastic Beanstalk configuration that allows me to deploy a .NET Windows Service but without deploying a web application.
I have just read this blog post which explains how to use .ebextensions to deploy a Windows Service alongside your web application, but is there a scenario for which the .ebextensions can be run without deploying a Web Deploy package for a web application?
Is my only option to create an empty web application that contains the .ebextensions directory and then deploy the Web Deploy package?
The Elastic Beanstalk FAQ mentions the ability to deploy non-web applications (here) and I have found a similar (unanswered) question on the AWS developer forums (here).
Update
Due to the lack of activity on this question and my inability to find any other information on the internet, I just assumed that the answer to this question is "No" (at least for now).
I ended up creating an empty web application and used that to deploy my Windows Service via the .ebextensions YAML config.
As a side note, I'd like to highlight this page from Amazon's documentation which I found to be a very helpful guide to creating those special config files.
Another Update
After implementing the approach mentioned above, I discovered that Elastic Beanstalk was not executing my .ebextensions scripts for new Beanstalk instances. As a result, the Windows Service failed to be installed when new instances were created. I had to jump through several more hoops to finally arrive at a scalable solution. Please let me know if you want the details of the final solution.
Ultimately, it just seems like Elastic Beanstalk wasn't meant for deploying scalable Windows Services.
Basic Solution
I'm not comfortable releasing the source code since it was not for a personal project, but here is the basic structure of my current deployment solution:
A custom EC2 AMI contains a 'bootstrap' program that runs on startup. The program does the following:
1.1. Download a 'zip' archive from a (configurable) 'deployment' S3 bucket
1.2. Extract the downloaded zip file to a temporary directory
1.3. An "install.bat" script is located/executed (the name of the script is also configurable). This script installs and starts the windows service.
The Elastic Beanstalk "Instance AMI" is set to the custom AMI with the bootsrap program (see: this article)
To deploy new code: upload the installation .zip archive (that contains the windows service and install.bat file) to the S3 bucket and terminate all EC2 instances for the Elastic Beanstalk application. As the instances are re-created the bootstrapping program will download/install the newly updated code.
Of course, if I were starting over, I would just skip using Elastic Beanstalk and use the standard AWS auto-scaling along with a similar deployment scheme. The bottom line is that if you don't have a web application, don't use Elastic Beanstalk; you're better off with the standard AWS auto-scaling.
New AWS Deployment Tools
Amazon recently announced several new code deployment/management services that seem to address deployment issues: http://aws.amazon.com/blogs/aws/code-management-and-deployment/
I have yet to use these new services (I'm not even sure if they've been released yet), but they look promising.
Since this question has been around for a while and still have no answer, but continues to draw interest, let me share my solution to a very similar problem - installing a Windows service on a EC2 instance. I'm not using Beanstalk though, since that service is designed more for quick deploy of web applications. Instead, I'm using directly CloudFormation which Beanstalk uses underneath to deploy resources related to the web application.
The stack expects existing VPC (our spans through several availability zones), a S3 bucket that stores all service build artifacts and a EC2 key pair. Template creates EC2 instance using Windows AMI and few other resources like IAM User with access keys and a working S3 bucket just for illustration of how to create additional resources that your service might need. Template also takes as a parameter the name of a zipped package with all service binaries and configuration files that's been uploaded on the build artifacts S3 bucket (we use TeamCity build server that makes that for us, but you can create and upload the package manually of course). When you build new version of the service, you simply create new package (for example service.v2.zip), update the stack with the new name and the service will be updated automatically. Template contains ids of AMIs in 4 different regions, but you can always add other regions if you wish. Here's the stack template:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "Service stack template.",
"Parameters": {
"KeyPair": {
"Type": "String",
"Default": "MyDefaultKeys",
"Description": "Name of EC2 Key Pair."
},
"ServicePackageName": {
"Type": "String",
"Default": "service.zip",
"Description": "Name of the zip package of the service files."
},
"DeploymentBucketName": {
"Type": "String",
"Default": "",
"Description": "Name of the deployment bucket where all the artifacts are."
},
"VPCId": {
"Type": "String",
"Default": "",
"Description": "Identifier of existing VPC."
},
"VPCSubnets": {
"Default": "",
"Description": "Commaseparated list of existing subnets within the existing VPC. Could be just one.",
"Type": "CommaDelimitedList"
},
"VPCSecurityGroup": {
"Default": "",
"Description": "Existing VPC security group. That should be the ID of the VPC's default security group.",
"Type": "String"
}
},
"Mappings": {
"Region2WinAMI": {
"us-east-1": { "64": "ami-40f0d32a" },
"us-west-1": { "64": "ami-20601740" },
"us-west-2": { "64": "ami-ff4baf9f" },
"eu-west-1": { "64": "ami-3367d340" }
}
},
"Resources": {
"ServiceInstance": {
"Type": "AWS::EC2::Instance",
"Metadata": {
"Comment": "Install Service",
"AWS::CloudFormation::Init": {
"configSets": {
"default": [ "ServiceConfig" ]
},
"ServiceConfig": {
"files": {
"c:\\service\\settings.config": {
"source": { "Fn::Join": [ "/", [ "https://s3.amazonaws.com", { "Ref": "DeploymentBucketName" }, "deployments/stacks", { "Ref": "AWS::StackName" }, "templates/settings.config.mustache" ] ] },
"context": {
"region": { "Ref": "AWS::Region" },
"accesskey": { "Ref": "IAMUserAccessKey" },
"secretkey": { "Fn::GetAtt": [ "IAMUserAccessKey", "SecretAccessKey" ] },
"bucket": { "Ref": "BucketName" }
}
},
"c:\\cfn\\cfn-hup.conf": {
"content": {
"Fn::Join": [
"",
[
"[main]\n",
"stack=",
{ "Ref": "AWS::StackId" },
"\n",
"region=",
{ "Ref": "AWS::Region" },
"\n",
"interval=1"
]
]
}
},
"c:\\cfn\\hooks.d\\cfn-auto-reloader.conf": {
"content": {
"Fn::Join": [
"",
[
"[cfn-auto-reloader-hook]\n",
"triggers=post.update\n",
"path=Resources.ServiceInstance.Metadata.AWS::CloudFormation::Init\n",
"action=cfn-init.exe -v -s ",
{ "Ref": "AWS::StackName" },
" -r ServiceInstance --region ",
{ "Ref": "AWS::Region" },
"\n"
]
]
}
}
},
"sources": {
"c:\\tmp\\service": { "Fn::Join": [ "/", [ "https://s3.amazonaws.com", { "Ref": "DeploymentBucketName" }, "deployments/stacks", { "Ref": "AWS::StackName" }, "artifacts/Service", { "Ref": "ServicePackageName" } ] ] }
},
"commands": {
"Install Service": {
"command": "call c:\\tmp\\service\\install.bat",
"ignoreErrors": "false"
}
},
"services": {
"windows": {
"cfn-hup": {
"enabled": "true",
"ensureRunning": "true",
"files": [ "c:\\cfn\\cfn-hup.conf", "c:\\cfn\\hooks.d\\cfn-auto-reloader.conf" ]
}
}
}
}
}
},
"Properties": {
"ImageId": { "Fn::FindInMap": [ "Region2WinAMI", { "Ref": "AWS::Region" }, "64" ] },
"InstanceType": "t2.micro",
"KeyName": { "Ref": "KeyPair" },
"SecurityGroupIds" : [{ "Ref": "VPCSecurityGroup" }],
"SubnetId" : { "Fn::Select": [ "0", { "Ref": "VPCSubnets" } ] },
"UserData": {
"Fn::Base64": {
"Fn::Join": [
"",
[
"<script>\n",
"if not exist \"C:\\logs\" mkdir C:\\logs \n",
"cfn-init.exe -v -s ",
{ "Ref": "AWS::StackName" },
" -r ServiceInstance --region ",
{ "Ref": "AWS::Region" },
" -c default \n",
"</script>\n"
]
]
}
},
"BlockDeviceMappings": [
{
"DeviceName": "/dev/sda1",
"Ebs": {
"DeleteOnTermination": "true",
"VolumeSize": "40",
"VolumeType": "gp2"
}
}
],
"Tags": [
{ "Key": "Name", "Value": { "Fn::Join": [ ".", [ { "Ref": "AWS::StackName" }, "service" ] ] } }
]
}
},
"BucketName": {
"Type": "AWS::S3::Bucket",
"Properties": {
"AccessControl": "PublicRead"
},
"DeletionPolicy": "Retain"
},
"IAMUser": {
"Type": "AWS::IAM::User",
"Properties": {
"Path": "/",
"Groups": [ "stack-users" ],
"Policies": [
{
"PolicyName": "giveaccesstobuckets",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [ "s3:*" ],
"Resource": [ { "Fn::Join": [ "", [ "arn:aws:s3:::", { "Ref": "BucketName" }, "/*" ] ] } ]
}
]
}
}
]
}
},
"IAMUserAccessKey": {
"Type": "AWS::IAM::AccessKey",
"Properties": {
"UserName": { "Ref": "IAMUser" }
}
}
}
}
As you can see, after copying the artifacts, we execute install.bat batch file (included in the zip file) that will move the files to the correct location and register the service. Here is the contents of the file:
#echo off
sc query MyService > NUL
IF ERRORLEVEL 1060 GOTO COPYANDCREATE
sc stop MyService
waitfor /T 20 ServiceStop
echo D | xcopy "c:\tmp\service" "c:\service\" /E /Y /i
GOTO END
:COPYANDCREATE
echo D | xcopy "c:\tmp\service" "c:\service\" /E /Y /i
sc create MyService binpath= "c:\service\MyService.exe" start= "auto"
:END
sc start MyService
Template also creates config file (from the settings.config.mustache which also resides on the artifacts bucket) containing information about the other resources that has been created for the service to use. Here it is:
<appSettings>
<add key="AWSAccessKey" value="{{accesskey}}" />
<add key="AWSSecretKey" value="{{secretkey}}" />
<add key="AWSRegion" value="{{region}}" />
<add key="AWSBucket" value="{{bucket}}" />
</appSettings>
You create and later update the stack either from the AWS web console or CLI.
And that's pretty much it. You can visit the AWS CloudFormation website to get more info about the service and how to work with templates.
P.S.: I realised that it would be better if I also share the template that creates the VPC. I keep it separate as I have one VPC per region. You can integrate it with the Service template if you wish, but that would mean that every time you create new stack a new VPC will be also created. Here is the VPC template:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "VPC stack template.",
"Mappings": {
"Region2AZ": {
"us-east-1": { "AZ": [ "us-east-1a", "us-east-1b", "us-east-1d" ] },
"us-west-1": { "AZ": [ "us-west-1b", "us-west-1c" ] },
"us-west-2": { "AZ": [ "us-west-2a", "us-west-2b", "us-west-2c" ] },
"eu-west-1": { "AZ": [ "eu-west-1a", "eu-west-1b", "eu-west-1c" ] }
}
},
"Conditions": {
"RegionHas3Zones": { "Fn::Not" : [ { "Fn::Equals" : [ { "Ref": "AWS::Region" }, "us-west-1" ] } ] }
},
"Resources": {
"VPC": {
"Type": "AWS::EC2::VPC",
"Properties": {
"CidrBlock": "10.0.0.0/16",
"EnableDnsSupport" : "true",
"EnableDnsHostnames" : "true"
}
},
"VPCSecurityGroup": {
"Type": "AWS::EC2::SecurityGroup",
"Properties": {
"GroupDescription": "Security group for VPC.",
"VpcId": { "Ref": "VPC" }
}
},
"Subnet0": {
"Type": "AWS::EC2::Subnet",
"Properties": {
"VpcId": { "Ref": "VPC" },
"CidrBlock": "10.0.0.0/24",
"AvailabilityZone": { "Fn::Select": [ "0", { "Fn::FindInMap": [ "Region2AZ", { "Ref": "AWS::Region" }, "AZ" ] } ] }
}
},
"Subnet1": {
"Type": "AWS::EC2::Subnet",
"Properties": {
"VpcId": { "Ref": "VPC" },
"CidrBlock": "10.0.1.0/24",
"AvailabilityZone": { "Fn::Select": [ "1", { "Fn::FindInMap": [ "Region2AZ", { "Ref": "AWS::Region" }, "AZ" ] } ] }
}
},
"Subnet2": {
"Type": "AWS::EC2::Subnet",
"Condition": "RegionHas3Zones",
"Properties": {
"VpcId": { "Ref": "VPC" },
"CidrBlock": "10.0.2.0/24",
"AvailabilityZone": { "Fn::Select": [ "2", { "Fn::FindInMap": [ "Region2AZ", { "Ref": "AWS::Region" }, "AZ" ] } ] }
}
},
"InternetGateway": {
"Type": "AWS::EC2::InternetGateway",
"Properties": {
}
},
"AttachGateway": {
"Type": "AWS::EC2::VPCGatewayAttachment",
"Properties": {
"VpcId": { "Ref": "VPC" },
"InternetGatewayId": { "Ref": "InternetGateway" }
}
},
"RouteTable": {
"Type": "AWS::EC2::RouteTable",
"Properties": {
"VpcId": { "Ref": "VPC" }
}
},
"Route": {
"Type": "AWS::EC2::Route",
"DependsOn": "AttachGateway",
"Properties": {
"RouteTableId": { "Ref": "RouteTable" },
"DestinationCidrBlock": "0.0.0.0/0",
"GatewayId": { "Ref": "InternetGateway" }
}
},
"SubnetRouteTableAssociation0": {
"Type": "AWS::EC2::SubnetRouteTableAssociation",
"Properties": {
"SubnetId": { "Ref": "Subnet0" },
"RouteTableId": { "Ref": "RouteTable" }
}
},
"SubnetRouteTableAssociation1": {
"Type": "AWS::EC2::SubnetRouteTableAssociation",
"Properties": {
"SubnetId": { "Ref": "Subnet1" },
"RouteTableId": { "Ref": "RouteTable" }
}
},
"SubnetRouteTableAssociation2": {
"Type": "AWS::EC2::SubnetRouteTableAssociation",
"Condition": "RegionHas3Zones",
"Properties": {
"SubnetId": { "Ref": "Subnet2" },
"RouteTableId": { "Ref": "RouteTable" }
}
},
"NetworkAcl": {
"Type": "AWS::EC2::NetworkAcl",
"Properties": {
"VpcId": { "Ref": "VPC" }
}
},
"AllowAllInboundTCPAclEntry": {
"Type": "AWS::EC2::NetworkAclEntry",
"Properties": {
"NetworkAclId": { "Ref": "NetworkAcl" },
"RuleNumber": "100",
"Protocol": "6",
"RuleAction": "allow",
"Egress": "false",
"CidrBlock": "0.0.0.0/0",
"PortRange": { "From": "0", "To": "65535" }
}
},
"AllowAllInboundUDPAclEntry": {
"Type": "AWS::EC2::NetworkAclEntry",
"Properties": {
"NetworkAclId": { "Ref": "NetworkAcl" },
"RuleNumber": "101",
"Protocol": "17",
"RuleAction": "allow",
"Egress": "false",
"CidrBlock": "0.0.0.0/0",
"PortRange": { "From": "0", "To": "65535" }
}
},
"AllowAllOutboundTCPAclEntry": {
"Type": "AWS::EC2::NetworkAclEntry",
"Properties": {
"NetworkAclId": { "Ref": "NetworkAcl" },
"RuleNumber": "100",
"Protocol": "6",
"RuleAction": "allow",
"Egress": "true",
"CidrBlock": "0.0.0.0/0",
"PortRange": { "From": "0", "To": "65535" }
}
},
"AllowAllOutboundUDPAclEntry": {
"Type": "AWS::EC2::NetworkAclEntry",
"Properties": {
"NetworkAclId": { "Ref": "NetworkAcl" },
"RuleNumber": "101",
"Protocol": "17",
"RuleAction": "allow",
"Egress": "true",
"CidrBlock": "0.0.0.0/0",
"PortRange": { "From": "0", "To": "65535" }
}
},
"SubnetNetworkAclAssociation0": {
"Type": "AWS::EC2::SubnetNetworkAclAssociation",
"Properties": {
"SubnetId": { "Ref": "Subnet0" },
"NetworkAclId": { "Ref": "NetworkAcl" }
}
},
"SubnetNetworkAclAssociation1": {
"Type": "AWS::EC2::SubnetNetworkAclAssociation",
"Properties": {
"SubnetId": { "Ref": "Subnet1" },
"NetworkAclId": { "Ref": "NetworkAcl" }
}
},
"SubnetNetworkAclAssociation2": {
"Type": "AWS::EC2::SubnetNetworkAclAssociation",
"Condition": "RegionHas3Zones",
"Properties": {
"SubnetId": { "Ref": "Subnet2" },
"NetworkAclId": { "Ref": "NetworkAcl" }
}
}
},
"Outputs": {
"VPC": {
"Description": "VPC",
"Value": { "Ref": "VPC" }
},
"VPCSecurityGroup": {
"Description": "VPC Security Group Id",
"Value": { "Fn::GetAtt": [ "VPCSecurityGroup", "GroupId" ] }
},
"Subnet0": {
"Description": "Subnet0 Id",
"Value": { "Ref": "Subnet0" }
},
"Subnet1": {
"Description": "Subnet1 Id",
"Value": { "Ref": "Subnet1" }
},
"Subnet2": {
"Description": "Subnet2 Id",
"Condition": "RegionHas3Zones",
"Value": { "Ref": "Subnet2" }
}
}
}
From my own experience, implementing anything using ebextensions significantly adds to the elapsed time for deployment. So much so that it can take up to 15 minutes for an instance to spin up when auto-scaling. Almost defeats the purpose.
In any event, be sure to configure the Auto Scaling Group's "Health Check Grace Period" property to something significant. For example, we use 900 (i.e. 15 minutes). Anything less and the instance never passes the health check and the scale up event fails; which makes for an unending series of attempts to scale up.

Categories

Resources