Articles for the Month of November 2016

Downloading Power BI Reports and the Impact on Report Creation Process

savepowerbireportWhile the official announcement has not been released, on November 24th the Power BI feature to Download report became available. It is now possible to download Power BI Reports from the Power BI Service. To access the feature, click on the dropdown file menu of a report, and there you will see the option to download a report. If your report was not created recently, this feature may not be enabled. If the report was created November 24th or later, the download menu item should be enabled. After the download option is selected, a message pops up letting you know the report is being downloaded. The message is accurate as it took a good three plus minutes for the report to be downloaded.

savepowerbirptmessage

There are a couple of points worth mentioning regarding the downloading feature. Downloading is for reports, not dashboards or datasets. **UPDATE: on 11-29 Microsoft released the information about the download feature. It can be turned off if on the Admin Portal exporting data is turned off. Unfortunately there isn’t a way to just turn off downloading the reports, as that one option controls both downloading reports and data.

Impact of Report Downloading on Templates and Content Packs

Power BI’s report download feature has the ability to modify the development process in Power BI as it is now possible to use Power BI Desktop and provide the ability to create report versioning. Unfortunately, a report created on the service, cannot be downloaded as only reports which originated as desktop files can be downloaded. This is a problem as these files cannot be sent to SharePoint Online to provide versioning. One has to hope no one broke a report created on the service as there is no method to rollback changes to a previous because no file exists. This limits the use of Organizational Content Packs. A Content Pack is designed to be a starting point for report development as it contains everything needed to create a report, including a connection to a dataset, plus dashboards and perhaps a report or two to serve as a style guide. Content Packs are accessed by the arrow at the bottom of the Power BI Menu or by selecting Get Data, then selecting My organization to use content packs created for local data access.

Now that anyone can use a report which has been uploaded to the service as a starting point for a new report, there may be a decreased use of the template feature. Any report created on the desktop can be saved as a template, by selecting save on a desktop report file and changing the file type to a template. Unfortunately, templates do not contain links to the datasource used. The person creating the report must determine what data to use and if there was a dataset presently used which is refreshing the data, or create a dataset and it’s respective refresh features as part of creating a report.  Content packs provide the connections to datasets, but since the reports cannot be saved as a file for versioning, this feature is not often used instead of templates. Downloading the file and then modifying it is does resolve the issue as the starting point is then a working report with a connection to an existing dataset.

Personally I have been waiting for Microsoft to release the download feature for a long time and am very happy to be able to use it. I hope that an some point the download feature will be expanded to reports created on the service as this feature will  improve the report creation process.

 

Yours Always

Ginger Grant

Data aficionado et SQL Raconteur

Admin and Auditing in Power BI

On October 24, 2016 Microsoft has created a new access right in Office 365, the Power BI Admin. This means that no longer does one have to be a Office 365 Administrator, a role with a ton of rights which have nothing to do with Power BI to be a Power BI Administrator. To be a Power BI Adminstrator, the Office 365 Adminstrator needs to run a PowerShell script to make a user a Power BI Administrator. Just change the email address and run this script.

Add-MsolRoleMember -RoleMemberEmailAddress “user@useremailaddress.com" -RoleName "Power BI Service Administrator"

After running this script you will almost have everything you need to be a Power BI Administrator. The one thing that the newly minted Power BI Administrator cannot do is review the Audit Logs.

Audit Logs in Power BI

The Audit Logs are the third menu item in the Power BI Admin Portal. As you can tell by looking at a copy of the screen below, Audit Logs are not really part of Power BI. Yes the ability to log all of the content in Power BI exists in the Audit Logs, but so does the ability to review the audit logs for things like Exchange Mailbox Activities and User Administration Activities.

Power BI Admin Screen

If the Office 365 Administrator has granted a user Power BI Administration rights, this is what the newly minted Power BI Administrator will see when trying to access any search activities. It appears that you the user has rights, until that user tries to do anything on the screen. At that point, this error window appears.

error-message-for-power-bi

Granting a Power BI Administrator the ability to view Power BI Audit Logs

To view audit logs, additional Office 365 access features are required. I did not figure out a way to turn on the audit feature unless one already was an Office 365 Administrator. If you want to view the audit logs, the first step should be to find out if they are turned on, because the steps I am going to show won’t do that. Within the Office 365 Admin center, go to the Exchange admin center and select permissions.

office365portal

Select the user and then add two roles Compliance Management and Organization Management. It is also possible to create a new role with Compliance Manager rights and add the Audit Logs and View Only Audit Log permissions, since those are the required permissions. None of this has anything to do with Power BI, which means when the permissions granted, users will have the ability to see all of the audit logs. After the new role permissions have been applied, the error message will go away and users will have the ability to see activity logged in Power BI as well as all of the Power BI Administration features.

 

Yours Always

Ginger Grant

Data aficionado et SQL Raconteur

Using Power BI to Display R Visuals

 Microsoft has in the past few months increased the support for R Visuals in Power BI. In the July release, Power BI Service started displaying interactive visuals and in October, six R visuals were made

rvisualsavailable in Power BI which do not require any knowledge of R. Six visuals is a far step away from the wide number of visuals available in R. According to the Comprehensive R [CRAN] as of today there are 9557 packages available for R, and that does not count all of the packages on GitHub.

R Packages Supported in the Power BI Service

According to a September 8th post on the Power BI Blog, there are currently 352 R packages supported in the Power BI Service. For those of you wondering if your favorite R library is included (yes dplyr, ggplot2 and cluster are there) I have included the list of packages here.

abc abc.data acepack actuar ade4 AdMit andrews
aod apcluster ape aplpack approximator arm arules
arulesViz assertthat BACCO BaM BAS base64enc bayesm
bayesQR bayesSurv BayesTree BayHaz bbemkr BCBCSF BCE
bclust BenfordTests bisoreg bit bit64 bitops BMA
Bmix BMS0 bnlearn, boa boot bootstrap bqtl
BradleyTerry2 brew brglm, bspec bspmma BVS C50
calibrator car caret, catnet caTools cclust class
clue cluster coda coin CORElearn corpcor corrplot
crayon cslogistic cubature cvTools data.table date dbscan
deal deepnet deldir dendextend DEoptimR deSolve DiagrammeR
dichromat digest dlm DMwR dplyr DPpackage dse
dtw e1071 earth EbayesThresh ebdbNet ellipse emulator
ensembleBMA entropy EvalEst evaluate evdbayes exactLoglinTest expm
extremevalues FactoMineR factorQR faoutlier fBasics fields filehash
fitdistrplus flashClust FME fmsb foreach forecast Formula
fracdiff fTrading futile.logger gam gamlr gclus gdata
gee genetics geoRglm geosphere ggdendro ggmcmc ggplot2
ggthemes glmmBUGS glmnet gmodels gmp gnm GPArotation
gridBase gridExtra growcurves grpreg gss gsubfn gtable
gtools haplo.stats hash hbsae hdrcde heavy HH
HI Hmisc HSAUR ifultools intervals ipred irlba
irr iterators jpeg kernlab KernSmooth KFKSDS kinship2
kknn klaR knitr labeling lars lattice latticeExtra
lava lavaan lazyeval leaps LearnBayes LiblineaR LICORS
limSolve lme4 lmm lmtest locfit locpol LogicReg
lsa lubridate magic magrittr mapdata mapproj maps
maptree MASS MasterBayes Matrix matrixcalc MatrixModels maxent
maxLik mboost mda memoise mi mice microbenchmark
mime misc3d miscTools mitools mixtools mlbench mnormt
MNP modeltools mombf monomvn MSBVAR msm multcomp
munsell mvtnorm NbClust ncvreg nlme nloptr NLP
NMF nnet nnls nortest numbers numDeriv OutlierDC
pacbpred party partykit PAWL pbivnorm pcaPP pdc
PerformanceAnalytics plotmo plotrix pls plyr png polynom
predmixcor PresenceAbsence profdpm proto proxy pryr pscl
psych qap qdapRegex quadprog quantreg qvcalc R.oo
.utils ramps RandomFieldsUtils RColorBrewer Rcpp RcppArmadillo RcppEigen
registry reliampo relimp reshape reshape2 RGraphics rjson
RJSONIO Rmpfr rms robustbase ROCR rpart.plot rrcov
rscproxy RSGHB RTextTools rworldmap SampleSizeMeans SampleSizeProportions sbgcop
scales scatterplot3d sciplot segmented seriation setRNG sfsmisc
SIS SixSigma skmeans slam smoothSurv sna snow
SnowballC snowFT sp spacetime spam SparseM spatial
spBayes spikeslab splancs spls spTimer sqldf sROC
stabledist stabs stepPlr stringdist stringr stsm stsm.class
survival tau tcltk2 tensorA textcat textir tfplot
TH.data tidyr timeDate timeSeries tm topicmodels tree
tseries tsfa tsoutliers TSP vcd VennDiagram VGAM
VIM whisker wordcloud xgboost XML xts yaml
zipfR zoo

 

 

I hope that some day that this list becomes much longer, but it is a good start. If your company has lots R visuals and you wish to migrate them to Power BI, chances are some of the libraries you are using are not here. If you are interested in having your library added to the list of 352, go to the Ideas page of Power BI and request that your library be added, as Microsoft I know looks at this page to determine what to release in the future. Someone has requested that igraph be added, and since it hasn’t received a lot of votes yet (hint) it is probably low on the priority list.

Release Plans for Additional R Library Support in Power BI

In reviewing the Power BI documentation online, it is not known what Microsoft’s plans are for future R packages. Does Microsoft plan on adding a lot more in the future? Will new libraries be added as part of upcoming releases? All questions where the web presently does not provide answers.

The current package support in Power BI does provide the R developer with a method to more securely and widely distribute R visualizations. In my upcoming posts I will review how to use the six R visualizations which do not require R and also how to use R with Power BI.

Yours Always

Ginger Grant

Data aficionado et SQL Raconteur

Questions and Answers on Machine Learning with R

opensourcerlogoRecently, I did a webinar on Machine Learning and R. I received a number of questions during the presentation. Due to time constraints, I was unable to answer all of them, so I have provided the Question and Answers here.

Question: Can I Use R in SQL Server to plot non-linear regression curves? We use IC50 and others in Michaelis-Menten kinetics for bio-chemical work.

R running on SQL Server provides the functionality of standard CRAN R packages with the additional capability to run the SCALER functions provided by SQL Server’s implementation of R. Any other functionality performed in R can therefore also be performed on SQL Server. Like all R code, you will need to install the required R libraries in the appropriate subdirectory. The directory is something like C:\Program Files\Microsoft SQL Server\MSSQL13.<instancename>\R_SERVICES\library

Question: Can you execute/invoke an external R script from T-SQL?

Yes. With SQL Server 2016 it is possible to run an R script from within T-SQL by using the sp_execute_external_script command. This link should help you with more detailed information on how to execute R scripts within T-SQL

Question: Is it possible to run R processes in diffrent boxes other than SQL Server itself for scalability reasons?

You have the option of installing the R Server on another server. Just keep in mind that you do have to account for the additional overhead of moving all the data over the network, which needs to weigh in on your decision to move processing to a different server.

Question: Can we join data generated from an R script to SQL data directly or does it have to be inserted into a table first?

Data generated from an R script interacts with the data the same way other data in a stored procedure does. It is not necessary to write the data to a table for the data to be processed.

Question: When would you use R Server versus R on SQL Server?

R on SQL server is an implementation of R server. When running R on SQL Server, R Server is running.

Question: What is the maximum file size I can load on R server?

R Server uses SCALR which provides the ability to not only use memory but also use disk storage, providing nearly limitless capability for file size processing.

Question: Is knitr integrated or can it be integrated into R for SQL Server?

While you can install the knitr library on SQL Server to implement the functionality provided, there is no direct integration path for incorporating the wide functionality knitr provides within SQL Server. For more information about knitr, please see this link.

Question: Does MS R Open have all the functionalities of CRAN R? Does it support all packages?

MS R Open is fully CRAN R compliant. All code which runs in CRAN R can run in MS R Open and all packages are supported. Since MS R Open was rewritten to use the Intel Math Kernel Libraries, it provides technical advantages to CRAN R. MS R Open is partially Multi-threaded, instead of single threaded like CRAN R, and is up to 38% faster

Question: Do you have a link for the Microsoft Machine learning cheat sheet?

Sure. The link can be found here.

Question: Why Use Microsoft R Open over other Analytics Tools such as Open R, Python, Matlab?

Microsoft R Open is fully CRAN R compliant, and also multi-threaded and faster, providing a clear benefit over CRAN R. Python is a great tool for data analytics, but unlike R, it is not designed solely for statistically analysis but has a wider functionality scope. R is focused solely on providing statistical data analysis. Matlab is a great tool, but given its complicated user interface and high licensing cost, many users may gravitate towards R, as it is free and there is a lot of good support for learning R available online as well.

Question: Can R be used with older versions of SQL Server or only 2016?

While it is possible to use R with other versions of SQL Server using tools such as RODBC, the R Server and full integration of R in SQL Server is only available in SQL Server 2016.

Question: Do I need all R server and R client and Microsoft R Open to be installed to run R from SQL Server?

To run R on SQL Server, R Server needs to be installed. For more information on how to install R on SQL Server please see this link. To connect to the server and use the SCALR functionality, the client machine will need to have R Client installed.

Question: What are the benefits of Microsoft Standalone R Server?

R Standalone Server is available to connect to data stored in HDFS and Teradata in addition to SQL Server. To incorporate data from other data sources and provide the ability to process more data than one has memory, it might make a lot of sense to deploy a standalone R Server.

Question: Are there any memory limitations in R when handling big data?

R is designed to run in memory, meaning if you have more data than memory you may run out of memory. When running using the SCALR functions, memory and disk are used to provide nearly limitless ability to process data.

Question: Can one call R from Python?

While it may be possible to create Python code which calls R, I am unaware of the syntax. This functionality would not be supported in the context of SQL Server.

Question: Can you use R charts in SSRS?

There are two different ways to incorporate R charts in SQL Server. Since a stored procedure can provide and R visualization, it is possible to use SSRS to call a stored procedure which would create a graphic to be rendered in SSRS. Also since Microsoft announced the incorporation of Power BI with SSRS, and Power BI provides the ability to create R graphics, incorporating Power BI which renders R would be another way to use charges in SSRS

Question: Any classes like “R for the nervous DBA” that you can recommend? I have a statistician who is running R on my SQL server.

I do not know of any R classes specifically designed for SQL Server DBAs, but I am working on a post for additional configuration information which you might find helpful. Please check back in a few weeks for updates.

Question: What is a Pirate’s favorite language?

RRRRRR Mad Props to Phillip for sending me a pirate joke. I love pirate jokes. I feel compelled to tell a few when I am presenting R and I have been told all of mine are really bad.

If you have any additional questions about SQL Server and R or any pirate jokes, please comment on this post or send me a message via twitter @desertislesql.

 

Yours Always

Ginger Grant

Data aficionado et SQL Raconteur