rsync - Syntax for Copying Data Between Two QNAP NAS Devices

 

Intro:

A few weeks ago I had a QNAP box shit the bed. One of the drives in the 8 disk array had a bad sector while another drive was throwing unrecoverable read errors. I needed an immediate way to copy the data to a new QNAP NAS since I could not get access to the SMB shares. When a disk or two fails based on the RAID level, the array falls into what QNAP calls, "Read-Only" mode. The array cannot be written to at this point which poses a problem if this is the first pool and your applications are installed here.

None of the GUI applications like "Hybrid Backup Sync 3" or "File Station" were working. Rsync is now the only option. Thankfully "Hybrid Backup Sync 3" was installed on the source box prior to the drives failing or I don't think the rsync service would have been active (there's a toggle to turn it on in HBS3). There is a way to stop and start all services but not individual ones on these boxes from what I can tell. It really wouldn't have mattered in this case since the applications cannot write temporary data to the pool since it's in a read-only state.

Rsync on QNAP devices seems to be a customized version. They're running "version 3.0.7 protocol version 30" dated 2009. According to rsync's wiki, the current stable version at the time of writing (09/08/23) is 3.2.7. So technically we're using a really outdated copy which may induce errors.

I have a 10Gbps fiber link between these two QNAP boxes using (2) Silicom Intel 82599ES Dual-Port SFP+ cards. (there's an interesting article coming about modifying the 82599 EEPROM of Intel based X520 cards to use any brand fibre transceiver, not just Intel branded ones). The source qnap system is in "Read-Only" mode so the copy operation is being slowed to an average of 150-250 MB/s. There's also other issues with one of the new disks appearing offline which may contribute to this however focusing on getting the data off this box asap is the priority here. If another disk fails I'm screwed.

So far I've copied about 70 TB over this link at those speeds. Painfully slow. At one point while reading and failing on a file, the read speed dropped to around 30 MB/s. Obviously there's an issue on the disk's surface in that area and that drive is destined for the scrap pile since it's out of warranty.

With the exception of that one small file which I had a backup for, everything else copied without incident taking approximately a week at those speeds.

rsync Syntax:

Here's the syntax I used to copy data from a TVS-871 (QTS) to a new TVS-h874 (QuTS hero):

rsync --progress --protect-args -avhro "/share/CACHEDEV1_DATA/<INSERT FOLDER HERE>/" user@172.16.0.3:"/share/ZFS19_DATA/<DESTINATION FOLDER>/"

Dry-Run:

To do a dry-run without copying anything, add an "n" to the beginning or end of "-avhro". This is good for determining the folder size prior to the actual copy operation.

Excluding Folders:

If you need to exclude folders create a file named, "pattern.txt" and add the excluded folders:

1. "vi pattern.txt"
2. Press "Insert" key to begin editing file.
3. Add one folder name per line i.e.,
• @Recycle
• .@__thumb
• .streams
4. ":wq!" to save and exit.
1. ":q!" to exit without saving.

Run the following command to invoke "pattern.txt" along with rsync.

rsync --progress --protect-args -avhro --exclude-from="/root/pattern.txt" "/share/CACHEDEV1_DATA/<INSERT FOLDER HERE>/" bob@172.16.0.3:"/share/ZFS19_DATA/<DESTINATION FOLDER>/"

Access Shared Mailbox Only via Outlook

 

Intro:

I have a client who changes employees every year and they need to access a single mailbox and not their own. For example they have, "legal@xyz.com" and need 3 people send/receive as that email address and not their own. To do this we'll use what's known as a "shared mailbox".

We need to add users first then add the shared mailbox. I guess you could do it the other way around but without a user how would you access the shared mailbox?

*I'm doing this as a Microsoft Partner so some of the steps may need to be skipped or omitted if you don't have a Partner account.

Process to Add Users:

Login to https://admin.microsoft.com/ or https://admin.microsoft.com/Adminportal/Home#/tenants (double check the tenant you're about to modify).

Select the tenant you wish to modify

Show All --> Users --> Active Users

Add a user --> Name, email address, select license (get from supplier), Roles = user, fill out profile info --> Finish Adding

Process to Add a Shared Mailbox:

Login to (https://admin.exchange.microsoft.com/) or (https://partner.microsoft.com/en-us/dashboard/home). If logging in as a Partner go to Customers --> Select customer --> Select Exchange under Administer Services. You should be taken to (https://admin.exchange.microsoft.com/) and your tenant should be stated near the top left with two opposing arrows to the right of the customer name.

Recipients --> Mailboxes

The user you create previously should be listed here.

Add a shared mailbox

Enter display name, email address, leave alias blank --> Create

Add users to this mailbox --> Add members --> Select members you want to have view/send/receive access --> Save.

Add delegate permissions? Yes.

(To double check this --> Click on shared mailbox in question and on the flyout click, Delegation. "Send as" and "Read and manage (Full Access)" need to contain the user you want to have sending/receiving mail as this mailbox.) You will receive errors if this delegation is not selected.

You'll see, "It might take up to 60 minutes for the change to be effective in Outlook and OWA." or some other time frame. So far I've seen 5 minutes on a Saturday night and 60 minutes on Sunday. Who knows.

Refresh the "Manage Mailboxes" page.

You should see the new shared mailbox.

Currently shared mailboxes that do not have an license attached to it such as Exchange Online Plan 2 are limited to 50gb. See here: https://learn.microsoft.com/en-us/microsoft-365/admin/email/about-shared-mailboxes?view=o365-worldwide

As for the space, here's some context. It took me since the beginning of Gmail in 2004 until today, 2023, to use approximately 14gb of space consisting of emails only. There's the occasional photo or two but mostly text based emails. I use every day for a variety of things. 50gb is plenty for a small business that mainly does text based emails.

Access Only This Shared Mailbox via Outlook:

In this instance this is only an Exchange Online plan with email only. There is no Microsoft 365 (formerly O365) account with Word, Excel, Outlook, etc baked into the license.

Launch Outlook on the PC. If you blew out the profiles it'll ask you for a new profile name. Just name it "default".

Enter the email address for the shared mailbox.

Click "Advanced Options" --> "Let me set up my account manually" --> Connect.

Click on "Microsoft 365". At the time of writing this icon is a red/orange/pink square.

Click "Sign in with another account" in the single sign on box.

Type in the user's actual email --> Next.

Type in the user's password --> Sign in.

Click "Skip for now (14 days until this is required)". This will be replaced with MFA per user. Be sure to set up MFA for your user or have them do it. Use Authy.

Stay signed in to all your apps? Since we only want to sign in to the Office applications and not the entire OS as this Microsoft account, click, "No, sign in to this app only" near the bottom in blue --> OK --> Done.

Exchange Account Settings (verify shared mailbox is at the top of this box) --> Move slider to the right to download all email --> Next --> Uncheck "Set up Outlook Mobile on my phone, too" --> Done.

The shared mailbox is now the only mailbox signed in and the user can continue on. Make sure to send/receive test email to verify the delegation section set above is working.

My Notes:

Process to clear out old Outlook profiles and start fresh:

Outlook 2021 --> C:\Program Files (x86)\Microsoft Office\root\Office16 --> OLCFG.EXE --> Show Profiles --> Select a profile name --> Remove

Delete anything here if all profiles have been deleted: C:\Users\XXXXXX\AppData\Local\Microsoft\Outlook

Start at (Access Only This Shared Mailbox via Outlook) to add a new mailbox from scratch.

👽

Windows - Reset TCP/IP Stack

 

Intro:

This is just a quick list of commands you can use to reset the TCP/IP stack in Windows.

Commands (CMD run as Admin):

ipconfig /flushdns

netsh winsock reset

netsh int ip reset

netsh interface ipv4 reset

netsh interface ipv6 reset

netsh interface tcp reset

netsh int reset all 

nbtstat -R

nbtstat -RR

netsh interface tcp set global autotuninglevel=disabled

netsh advfirewall reset

PowerShell (run as admin):

Get-NetAdapter | Restart-NetAdapter

Conclusion:

I'll add to this list as I either remember forgotten commands or I come across new ones.

👽