Vertigo Is a Bitch 😀🤨😵🥴🤢

03 - Cloud Backup - Wasabi - Setup MFA, Bucket, Sub Folders, Users, and Policy Setup - Part 3 of 5

Intro:

This is Part 3 of 5 on how to setup and use Duplicati with Wasabi's Hot Cloud Storage.

This Article Explains How To:

Enable Multi-Factor Authentication on the Sub-Account

Create a Bucket(s) (each pc being backed up goes in its own bucket)

Create two folders inside the bucket. One for the Duplicati files and one for a backup of the Duplicati database.

Create a user (each pc being backed up should have its own user)

Create and apply a policy to the user (not the bucket)

Test the Wasabi policy

Enable Multi-Factor Authentication on the Sub-Account:

1. Login to the Wasabi Console at: https://console.wasabisys.com/

2. The most important thing to do immediately after setting up a sub-account is to turn on Multi-factor authentication. It can be found in settings. Click on the person icon in the top right then click Settings.

3. Click on MFA Settings:

4. Leave, "MFA Recovery Codes" disabled since you can reset the root user's password from the WACM if you forget it. Use something like, Authy, to save this token. Enter one code then wait 30 seconds for another code to be generated and enter that code. Click on, "Activate Virtual MFA" when finished.

5. If the multi-factor authentication setup was successful you should see the following under MFA Settings:

Pay attention going forward.

Things to Accomplish:

1. Create a bucket with the following naming convention: "duplicati-pcname" where "pcname" is the actual name of the computer.

2. Create two folders inside the bucket. One is for the Duplicati backups of files on the computer and the other folder is for a backup of the database. This will come into play later when we do the restore for very large backups (100gb and up). Instead of pointing Duplicati to the files and saying recover them, which will be slow as it builds a new database, we can copy the existing database over and it'll accomplish this process much faster.

3. Create a Wasabi user and save the Access and Secret key (one user and one set of keys per computer to be backed up)

4. Create a policy and apply it to the Wasabi user.

If you remember from the previous article our sample customer was, "John's Seafood Shack". This customer is located at 1500 Green Road and the email we setup for the Wasabi account is, "cx-jss1500@xyz.com".

Since we're logged into the Wasabi console as the root user of John's Seafood Shack, we need to create a simple bucket naming convention when adding new computers to backup (replace "pcname" with the actual name of the computer):

"duplicati-pcname"

Wasabi buckets need to be globally unique in their system. If another Wasabi user is already using the bucket name you picked then the system will prevent you from creating that bucket. The odds of someone using the same naming convention and having the same customer as you are slim.

1 - Create a Bucket (one per pc to be backed up):

Before creating a bucket make sure you are editing the correct account! After logging in to the sub-account you can view this by clicking, "Settings" on the left then looking at the 3rd line down. It will say, "Account Name". I sent Wasabi a feature request for this, to identify somewhere on the main screen telling you which account you are logged into so you don't screw up an account you've accidentally signed into.

*The bucket name is globally unique to Wasabi.com and uses all lowercase characters.

1. On the main screen after logging in click, "Create" in the top right corner. Create a bucket with the following naming convention, "duplicati-pcname" where "pcname" is the actual name of the pc. Pick the region that you want to store the data in. In this case we will select, "N. Virginia us-east-1 (s3.wasabisys.com)".

When done click, "Create Bucket".

2. You should see the following after bucket creation:

Create Duplicati Folders Inside the Bucket:

1. Click on the new bucket name then on the next page click, "Create Folder" in the top right. In the screenshot below we are already in this bucket. Proceed to step 2.

2. Create the following folders in the root of the bucket:

duplicati
duplicati-database

3. It should look like this when you're done:

Create a User:

1. From the menu on the left, click, "Users".

2. Create a new user with the same name as the bucket name, "duplicati-desktop-488gd38". Select "Programmatic (create API key)". DO NOT SELECT, "CONSOLE"! Click, "next" when done.

3. Skip group creation for now:

4. Skip attaching a policy for now:

5. Verify the username is correct, API is selected, and console access is set to NO. Click, "Create User" when done.

6. After you click, "Create User" the prompt will change and it will have a Secret Key and Access Key. These are the keys that will be used to perform the backups. Make sure you download these. If you lose them you need to destroy the existing keys and regenerate a new set for the user.

7. Once you've downloaded them you can close the above box and proceed.

Create a Policy:

The policy we're about to create will give the user the ability to write to two folders, "duplicati" and "duplicati-database". For every new folder you create inside a bucket, you need to add a new policy for it or review the code below and duplicate the blocks of code that are responsible for the write access.

Here's the 3 blocks that need to be duplicated if you add another folder (only one policy at a time can be enabled on Wasabi's end). Pay attention to the curly brackets and make sure you properly close each block of code:

AllowRootAndHomeListingOfCompanyBucket
AllowListingOfUserFolder
AllowAllS3ActionsInUserFolder

1. On the menu to the left click on, "Policies" then click, "Create Policy" on the top right.

2. Create a new policy with the same name as the user and the bucket, "duplicati-desktop-488gd38". Copy the following into the description, "Allows Duplicati user to read and write in its own bucket and nothing else".

3. Before you copy the below code into the, "Policy Document" section as seen in the screenshot above, we need to change some variables.

Using our bucket name of, "duplicati-desktop-488gd38" we're going to replace every instance of, "duplicati-pcname" below with our bucket name, "duplicati-desktop-488gd38". There are 8 instances that need to be replaced.

You don't have to mess with the policy itself as I've already written that out to work with the two sub folders, "duplicati" and "duplicati-database".

START OF TEMPLATE:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95 {
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowUserToSeeBucketListInTheConsole",
      "Effect": "Allow",
      "Action": [
        "s3:ListAllMyBuckets",
        "s3:GetBucketLocation",
        "s3:GetBucketCompliance"
      ],
      "Resource": "arn:aws:s3:::*"
    },
    {
      "Sid": "AllowRootAndHomeListingOfCompanyBucket",
      "Effect": "Allow",
      "Action": "s3:ListBucket",
      "Resource": "arn:aws:s3:::duplicati-pcname",
      "Condition": {
        "StringEquals": {
          "s3:delimiter": "/",
          "s3:prefix": [
            "",
            "duplicati"
          ]
        }
      }
    },
    {
      "Sid": "AllowRootAndHomeListingOfCompanyBucket",
      "Effect": "Allow",
      "Action": "s3:ListBucket",
      "Resource": "arn:aws:s3:::duplicati-pcname",
      "Condition": {
        "StringEquals": {
          "s3:delimiter": "/",
          "s3:prefix": [
            "",
            "duplicati-database"
          ]
        }
      }
    },
    {
      "Sid": "AllowListingOfUserFolder",
      "Effect": "Allow",
      "Action": "s3:ListBucket",
      "Resource": "arn:aws:s3:::duplicati-pcname",
      "Condition": {
        "StringLike": {
          "s3:prefix": "duplicati/*"
        }
      }
    },
    {
      "Sid": "AllowListingOfUserFolder",
      "Effect": "Allow",
      "Action": "s3:ListBucket",
      "Resource": "arn:aws:s3:::duplicati-pcname",
      "Condition": {
        "StringLike": {
          "s3:prefix": "duplicati-database/*"
        }
      }
    },
    {
      "Sid": "AllowAllS3ActionsInUserFolder",
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:DeleteObject"
      ],
      "Resource": "arn:aws:s3:::duplicati-pcname/duplicati*"
    },
    {
      "Sid": "AllowAllS3ActionsInUserFolder",
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:DeleteObject"
      ],
      "Resource": "arn:aws:s3:::duplicati-pcname/duplicati-database*"
    },
    {
      "Effect": "Deny",
      "Action": "s3:DeleteBucket",
      "Resource": [
        "arn:aws:s3:::duplicati-pcname",
        "arn:aws:s3:::duplicati-pcname/*"
      ]
    }
  ]
}

END OF TEMPLATE

Using our example it should now look like:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowUserToSeeBucketListInTheConsole",
      "Effect": "Allow",
      "Action": [
        "s3:ListAllMyBuckets",
        "s3:GetBucketLocation",
        "s3:GetBucketCompliance"
      ],
      "Resource": "arn:aws:s3:::*"
    },
    {
      "Sid": "AllowRootAndHomeListingOfCompanyBucket",
      "Effect": "Allow",
      "Action": "s3:ListBucket",
      "Resource": "arn:aws:s3:::duplicati-desktop-488gd38",
      "Condition": {
        "StringEquals": {
          "s3:delimiter": "/",
          "s3:prefix": [
            "",
            "duplicati"
          ]
        }
      }
    },
    {
      "Sid": "AllowRootAndHomeListingOfCompanyBucket",
      "Effect": "Allow",
      "Action": "s3:ListBucket",
      "Resource": "arn:aws:s3:::duplicati-desktop-488gd38",
      "Condition": {
        "StringEquals": {
          "s3:delimiter": "/",
          "s3:prefix": [
            "",
            "duplicati-database"
          ]
        }
      }
    },
    {
      "Sid": "AllowListingOfUserFolder",
      "Effect": "Allow",
      "Action": "s3:ListBucket",
      "Resource": "arn:aws:s3:::duplicati-desktop-488gd38",
      "Condition": {
        "StringLike": {
          "s3:prefix": "duplicati/*"
        }
      }
    },
    {
      "Sid": "AllowListingOfUserFolder",
      "Effect": "Allow",
      "Action": "s3:ListBucket",
      "Resource": "arn:aws:s3:::duplicati-desktop-488gd38",
      "Condition": {
        "StringLike": {
          "s3:prefix": "duplicati-database/*"
        }
      }
    },
    {
      "Sid": "AllowAllS3ActionsInUserFolder",
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:DeleteObject"
      ],
      "Resource": "arn:aws:s3:::duplicati-desktop-488gd38/duplicati*"
    },
    {
      "Sid": "AllowAllS3ActionsInUserFolder",
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:DeleteObject"
      ],
      "Resource": "arn:aws:s3:::duplicati-desktop-488gd38/duplicati-database*"
    },
    {
      "Effect": "Deny",
      "Action": "s3:DeleteBucket",
      "Resource": [
        "arn:aws:s3:::duplicati-desktop-488gd38",
        "arn:aws:s3:::duplicati-desktop-488gd38/*"
      ]
    }
  ]
}

*The policy pretty much says what it's doing and not doing. With the above policy the, "duplicati-desktop-488gd38" user cannot write to the root folder, "duplicati-desktop-488gd38" but you can write inside of the two folders sitting in the root bucket, "duplicati" and "duplicati-database".

*The user cannot delete the bucket itself but they can remove the files inside of the folders as well as delete the folders themselves. Looking for a way to stop this but I don't think it's possible because folders don't really exist as per the Amazon s3 specifications.

4. Once you are sure you've changed all 8 variables marked in light red from the template, copy the updated template and paste it into the, "Policy Document" section as seen below. If the policy is valid and has no syntax errors it will say, "Policy is valid". Once that happens click, "Create Policy" at the bottom. If not go back and make sure you didn't accidentally remove a character.

5. You'll see the following message after a successful creation and will be dropped back off at the policies page. Look closely and you'll see there are 11 policies but the page is only showing you policies 1-10. To see the one you just created you need to change, "Rows per page" to 25 near the bottom. *After you attach the policy to the user you can click on the icon below to filter out only policies applied to users:

Apply the Policy to the User We Created Earlier:

* Still on the Policies page notice there's a zero to the right of the policy name we just created in the column for, "Number Attached". This simply means the number of users that policy is attached to at the time of the page refresh.

1. Menu on the left, click on, "Users".

2. Click on our username, "duplicati-desktop-488gd38".

3. Select, "Policies".

4. Notice by default there are no policies attached to this user.

5. To add one, click to the right of the magnifying glass🔍and select the policy name, "duplicati-desktop-488gd38" we created earlier.

6. Note there is no save button on this page. After you select the policy it will show up as a bubble near the bottom of the policy tab.

At this point you're done. You can close the page.

Testing the Wasabi Policy:

1. Download Wasabi's tailored version of CloudBerry Explorer. It's free for Wasabi users and can only connect to Wasabi accounts. It's basically a stripped down version of CloudBerry Explorer. Scroll down the page and the download for the app is under, "Installation Instructions".

2. Install the application then open it.

3. We're going to use the Secret and Access Keys for the Wasabi user, "duplicati-desktop-488gd38". Do not use the root credentials for this test!

*Remember root can do anything including deleting the folders we created earlier. Our policy stops the sub users from doing this.

4. Open Wasabi Explorer.

5. File --> Wasabi

6. In the box that pops up enter the following (enter the credentials that were generated when you created a user earlier) :

Display Name = duplicati-desktop-488gd38
Access Key = Your access key
Secret Key = Your secret key
Enable - Use SSL

Click, "Test Connection". If everything works you'll get a green check mark like below. If you don't double check your credentials.

When done, press, "ok" on the above box, then, "close" on the box behind it labeled, "Registered Accounts".

7. Click on the, "Source:" dropdown and select, "duplicati-desktop-488gd38".

8. Double click on the bucket labeled, "duplicati-desktop-488gd38" below.

9. You should now see the two folders you created earlier, "duplicati" and "duplicati-database".

10. If you want to check to make sure the policy is correct, try to drag a small file from your Desktop into one of those folders. If you can upload and delete that file from the Wasabi Explorer then you're ready to go.

*Be careful since you can still remove the folders themselves.

TL;DR:

1. Log into https://console.wasabisys.com/

2. Settings --> MFA Settings --> Enable

2. Create a new bucket named, "duplicati-pcname" on "s3.wasabisys.com" --> Create Bucket

3. Create two folders inside this bucket:

duplicati
duplicati-database

4. Create a user named, "duplicati-pcname" --> Programmatic (Create API Key).

WAIT FOR THE KEYS TO BE GENERATED AND COPY THEM!!!

5. Create a policy using the above template and name it, "duplicati-pcname". Edit the 8 variables in notepad.

Use, "Allows Duplicati user to read and write in its own bucket and nothing else" as the description.

6. Apply the policy to the user. Users --> Pick Username --> Policies --> Start typing to find new policy --> Select (There's no save on this page, the policy is applied instantly).

7. Test using Wasabi Explorer

8. Update documents.

Next step is creating the backup job in Duplicati itself.

👽

I'm using http://hilite.me/ to reproduce the beautiful policy code above

02 - Cloud Backup - Wasabi Account Control Manager (WACM) - Sub-Account Creation - Part 2 of 5

Intro:

This is part 2 of 5.

*For each customer location we will create a new Wasabi sub-account along with a new email address.

*Individual Wasabi sub-accounts will allow us to correctly bill each customer location for the amount of data they use per month.

*The email address will be used to login to the Wasabi sub-account as well as send reports to itself from Duplicati.

Create Email Account (on your email server):

Gear icon --> Email Services --> xyz.com --> Email Accounts --> + Add
Create new email in the form of, "cx-NameOfBusinessAbbreviationAddressNumber@xyz.com".

Start with, "cx-"
John's Seafood Shack
1500 Green Road
"cx-jss1500@xyz.com"

Generate a 20 character password using numbers, letters, special characters, etc.
Done.

Create Sub-Account in WACM:

1. Login to "https://wacm.wasabisys.com/".

2. Create Sub-Account (blue pill icon on right side).

3. Create Sub-Account Details:

Name of actual customer location, ie: "John's Seafood Shack" or "John's Seafood Shack - Crab Key Island".
Email you created above
Generate a 20 digit password for the console login
Make sure, "Send password reset to sub account email" and "On Trial" are selected.
Change, "Trial Days" from "30" to "90" days to ensure you have enough time to test out the setup before committing the account to a paid one.
When done click on, "Create".
DOWNLOAD A COPY OF THE ACCESS AND SECRET KEY!!! (you will not be able to view the secret key later!)

4. On the Sub-Accounts page, under name, click on the new account name in blue on the left side then on the next page click on the tab labeled, "Account".

5. Fill out the customers contact information on the, "Account" tab under, "Contact Information":

6. Make sure you use the new email for both the "Contact" and "Billing" email.

7. When done be sure to click the blue update pill button in the top right corner.

8. Customer account creation is now done.

Proceed to step 3 of 5 to setup and configure MFA (multi-factor authentication), Buckets, Sub Folders, Users, and Policies.

TL;DR:

1. Create a new email account for the customer such as, "cx-jss1500@xyz.com" and generate a random 20 character password for the email.

2. Login to WACM (https://wacm.wasabisys.com/) with your MSP credentials.

3. Create a new sub-account using the customer's business name. If this is for a second physical location append, "- Actual Location Name" to the customer name. ie: "John's Seafood Shack" or "John's Seafood Shack - Crab Key Island".

4. Use email in form of, "cx-jss1500@xyz.com" and generate another random 20 character password for the WACM sub-account login.

5. Make sure, "Send password reset to sub account email" and "On Trial" are selected.

6. Change, "Trial Days" from "30" to "90" days.

7. Download a copy of the Access and Secret Keys.

8. On the sub-accounts page, click on the new customer location --> Account. Fill out contact section at bottom of page with customer address, etc.

9. Use, ""cx-jss1500@xyz.com" for Contact and Billing emails --> Scroll up and click Update.

10. WACM sub-account is ready to go, make sure to setup and enable MFA once inside the customer account.

👽

01 - Cloud Backup - Duplicati - Initial Setup & Config - 2021 - Part 1 of 5

Intro:

This is a multi-part article and is part 1 of 5.

I've been using Duplicati to backup to several site locations for quite some time to an Amazon compatible S3 service. Prior to deployment I strongly suggest you test out Duplicati's settings and make sure everything works the way you want it to. Once it's setup and working, it's fully automated. You should test a restore here and there to make sure it's working properly. You should also have a backup plan worked out with your customer, ie how many days to store, what to do with that data when those days are up, etc.

You also need to check your S3 storage provider and watch for fees as the amount of data you store will increase at some point. You can read more about the way Duplicati works here.

In short, Duplicati compresses your data into archives (default of 50mb), then every couple of days it cleans up the data archive files and needs to delete and recreate the archive. Your S3 provider may count that deleted data against your total storage space in a given month and or a minimum storage duration policy may apply.

Notes on Wasabi:

*I'm going to explain this article from a small MSP's point of view. Part 2 of X explains how to setup a Wasabi sub-account for each customer's physical location. Reach out to your Wasabi rep to have the WACM (Wasabi Account Control Manager) activated.

WACM gives you the ability to create accounts with trials starting at 30 days to get rolling without incurring charges while testing. You can extend this up to 90 days for your customer in increments of 30 days if they need more time. You can also promote an account from trial mode to paid mode within those 90 days. After 90 days the account becomes a paid account automatically.

Please have a look at this page to understand Wasabi's pricing model. You may also want to look at this page to understand how Wasabi's "minimum storage duration policy" works so the bill isn't a surprise at the end of the month.

Wasabi offers pay as you go and prepaid plans called, "Reserved Capacity Storage". For the latter you can pay up front for a portion of data at a lesser price. For the RCS payment model there is a minimum of 25 terabytes.

Requirements: