Shinylets us write client-side front-end code in R itself and also lets users write server-side script in R itself. More details on this package can be found here.
I keep meaning to learn Shiny and someday I will, just to prove to my intern that she’s not the only one here who can…
A thorny topic. Rolling back code is easy. Normally you can just redeploy the old binaries and you’re done. Databases are more difficult – because data. The existence of persistent data has two significant consequences:
You can’t just redeploy the entire database – you need to create a script to migrate your existing database from the existing (undesirable) state back to the old (desired) state. This probably isn’t an automated process.
You need to think about new data that has entered the database since the deployment. Do you want to keep it? Do you need to process it to get it into an older format? (E.g. if you need to undo a column split.)
Alex has some helpful tips for structuring database changes. Me, I just never look back…which is actually one of the strategies Alex talks about.
In 2017 we conducted our first ever extra-large, industry-wide survey to captured the state of data science and machine learning.
As the data science field booms, so has our community. In 2017 we hit a new milestone of reaching over 1M registered data scientists from almost every country in the world. Representing many different backgrounds, skill levels, and professions, we were excited to ask our community a wide range of questions about themselves, their skills, and their path to data science. We asked them everything from “what’s your yearly salary?” to “what’s your favorite data science podcasts?” to “what barriers are faced at work?”, letting us piece together key insights about the people and the trends behind the machine learning models.
Without further ado, we’d love to share everything with you. Over 16,000 responses surveys were submitted, with over 6 full months of aggregated time spent completing it (an average response time of more than 16 minutes).
Click through for a few reports. Something interesting to me is that the top languages/tools were, in order, Python, R, and SQL. For the particular market niche that Kaggle competitions fit, that makes a lot of sense: I tend to like R more for data exploration and data cleansing, but much of that work is already done by the time you get the dataset.
The general steps are
Set up your source query.
Pass the data through a Lookup for your Dimension with the missing results routed to a “No Match” output.
Insert those “No Match” rows into your Dimension using a SQL task – checking to make sure that this particular row hasn’t already been inserted (this is important).
Do another lookup using a “Partial Cache” to catch these newly-inserted members.
Use a UNION ALL transform to bring the existing and late-arriving members together.
Click through for more information and a helpful package diagram.
Now in the output above, you’ll notice a bolded line. In there, you can system that systemd receives a return code from SQL Server of status=1/FAILURE. Systemd is the parent process to sqlservr, in fact it’s the parent to all processes on our system. It receives the exit code and immediately, systemd initiates a restart of the service due to the configuration we have for our mysql-server systemd unit.What’s interesting is that this happens even on a normal shutdown. But that simply doesn’t make sense, return values on clean exits should return 0. It’s my understanding of the SHUTDOWN command, that it will cause the database engine to shutdown cleanly.
On the development side, there aren’t many differences between SQL on Linux versus SQL on Windows (aside from things which haven’t yet made the move); on the administration side, there are some interesting differences.
Top (in no particular order) Settings I have Implemented Through Model
Default Growth Settings
Query Store Settings
Read Committed Snapshot Isolation
Allow Snapshot Isolation
Auto Update Statistics Asynchronously
Click through for more information and to see how to make some of these changes.
Take note here that I am only querying the msdb database. There is nothing exceedingly top secret here – yet. Most DBAs should be extremely familiar with these tables and functions that I am using here.
What does this show me though? If I have a package that is being run via Agent Job in msdb, then the sensitive information needs to be decrypted somehow. So, in order to do that decryption the password needs to be passed to the package. As it turns out, the password will be stored in the msdb database following the “DECRYPT” switch for the dtutil utility. Since I happen to have a few of these packages already available, when I run this particular query, I will see something like the following in my results.
That’s a clever solution. I get the feeling that I should be a bit perturbed by how simple this is, but I don’t; the real sensitive data is still secure.