I went and looked on the server, and sure enough in Management Studio I saw one database named “FinanceDB” and a database named “[FinanceDB]”.
This was on a SQL 2008R2 instance, but as a test I created a database named [test] on my local SQL 2014 instance and sure enough it worked!
The source of the problem at the client was the LiteSpeed maintenance plan. Even though the backup task was set to backup all user databases, it wasn’t picking up the square-bracketed database.
I’d go a bit further and say that you should avoid everything but alpha-numeric characters and maybe underscore for databases, tables, views, and all other database objects.