But life is unpredictable, so I can announce that I have recently joined Oracle UK and will focus on promoting Oracle cloud technologies.
Oracle Again after 13 years!!!
Oracle R Enterprise installation
In this post I would like to install Oracle R Enterprise 1.4.0. version.
Note: I am going to install the tool (or option) on a linux server (OEL 6.3 64 bit) with Oracle DB 11.2.0.4., so only "server" configuration is described.
Step 1 - Oracle R 3.0.1
Let's start. According to documentation, Oracle R 3.0.1 is required for Oracle R Enterprise 1.4.0 (I think you can use the regular R, but Oracle strongly recommends to use their own version). Install the software using yum.
Check that everything works fine.
Step 2 - Oracle R Enterprise
Download the "server" file from OTN. Unzip the file and run install.sh. Don't forget to set ORACLE_HOME and ORACLE_SID.
Step 3 - Install packages
Download the Download the "supporting" file from OTN. Unzip the file and install the packages.
Step 4 - create a Database User
In this case, I used the demo_user.sh script from the server folder.
Just in case, provide the RQADMIN grant.
Step 5 - Test
Setting up the correct library paths and run demo.
Note: I am going to install the tool (or option) on a linux server (OEL 6.3 64 bit) with Oracle DB 11.2.0.4., so only "server" configuration is described.
Step 1 - Oracle R 3.0.1
Let's start. According to documentation, Oracle R 3.0.1 is required for Oracle R Enterprise 1.4.0 (I think you can use the regular R, but Oracle strongly recommends to use their own version). Install the software using yum.
[root@oel ~]# yum install R-3.0.1 Loaded plugins: refresh-packagekit, security Setting up Install Process Resolving Dependencies --> Running transaction check ---> Package R.x86_64 0:3.0.1-2.el6 will be installed --> Processing Dependency: libRmath-devel = 3.0.1-2.el6 for package: R-3.0.1-2.el6.x86_64 --> Processing Dependency: R-devel = 3.0.1-2.el6 for package: R-3.0.1-2.el6.x86_64 --> Running transaction check ---> Package R-devel.x86_64 0:3.0.1-2.el6 will be installed --> Processing Dependency: R-core = 3.0.1-2.el6 for package: R-devel-3.0.1-2.el6.x86_64 ---> Package libRmath-devel.x86_64 0:3.0.1-2.el6 will be installed --> Processing Dependency: libRmath = 3.0.1-2.el6 for package: libRmath-devel-3.0.1-2.el6.x86_64 --> Running transaction check ---> Package R-core.x86_64 0:3.0.1-2.el6 will be installed ---> Package libRmath.x86_64 0:3.0.1-2.el6 will be installed --> Finished Dependency Resolution Dependencies Resolved ======================================================================================================================================================================== Package Arch Version Repository Size ======================================================================================================================================================================== Installing: R x86_64 3.0.1-2.el6 ol6_addons 15 k Installing for dependencies: R-core x86_64 3.0.1-2.el6 ol6_addons 45 M R-devel x86_64 3.0.1-2.el6 ol6_addons 86 k libRmath x86_64 3.0.1-2.el6 ol6_addons 111 k libRmath-devel x86_64 3.0.1-2.el6 ol6_addons 20 k Transaction Summary ======================================================================================================================================================================== Install 5 Package(s) Total download size: 46 M Installed size: 81 M Is this ok [y/N]: y Downloading Packages Rdevel-3.0.1-2.el6.x86_64.rpm | 86 kB 00:00 Running rpm_check_debug Running Transaction Test Transaction Test Succeeded Running Transaction Installing : libRmath-3.0.1-2.el6.x86_64 1/5 Installing : libRmath-devel-3.0.1-2.el6.x86_64 2/5 Installing : R-core-3.0.1-2.el6.x86_64 3/5 Installing : R-devel-3.0.1-2.el6.x86_64 4/5 Installing : R-3.0.1-2.el6.x86_64 5/5 Verifying : libRmath-devel-3.0.1-2.el6.x86_64 1/5 Verifying : R-core-3.0.1-2.el6.x86_64 2/5 Verifying : libRmath-3.0.1-2.el6.x86_64 3/5 Verifying : R-devel-3.0.1-2.el6.x86_64 4/5 Verifying : R-3.0.1-2.el6.x86_64 5/5 Installed: R.x86_64 0:3.0.1-2.el6 Dependency Installed: R-core.x86_64 0:3.0.1-2.el6 R-devel.x86_64 0:3.0.1-2.el6 libRmath.x86_64 0:3.0.1-2.el6 libRmath-devel.x86_64 0:3.0.1-2.el6 Complete! [root@oel ~]#
Check that everything works fine.
[root@oel ~]# R Oracle Distribution of R version 3.0.1 (--) -- "Good Sport" Copyright (C) The R Foundation for Statistical Computing Platform: x86_64-unknown-linux-gnu (64-bit) R is free software and comes with ABSOLUTELY NO WARRANTY. You are welcome to redistribute it under certain conditions. Type 'license()' or 'licence()' for distribution details. Natural language support but running in an English locale R is a collaborative project with many contributors. Type 'contributors()' for more information and 'citation()' on how to cite R or R packages in publications. Type 'demo()' for some demos, 'help()' for on-line help, or 'help.start()' for an HTML browser interface to help. Type 'q()' to quit R. You are using Oracle's distribution of R. Please contact Oracle Support for any problems you encounter with this distribution. > q(); Save workspace image? [y/n/c]: n [root@oel ~]#
Step 2 - Oracle R Enterprise
Download the "server" file from OTN. Unzip the file and run install.sh. Don't forget to set ORACLE_HOME and ORACLE_SID.
[oracle@oel server]$ ./install.sh
Oracle R Enterprise 1.4 Server Installation.
Copyright (c) 2012, 2013 Oracle and/or its affiliates. All rights reserved.
Checking R ................... Pass
Checking R libraries ......... Pass
Checking ORACLE_HOME ......... Pass
Checking ORACLE_SID .......... Pass
Checking sqlplus ............. Pass
Checking ORACLE instance ..... Pass
Checking ORE ................. Pass
Current configuration
R_HOME = /usr/lib64/R
R_LIBS_USER = /u01/app/oracle/product/11.2.0.4/dbhome_1/R/library
ORACLE_HOME = /u01/app/oracle/product/11.2.0.4/dbhome_1
ORACLE_SID = orcl
Do you wish to install ORE? [yes]
Choosing RQSYS tablespaces
PERMANENT tablespace to use for RQSYS [SYSAUX]: USERS
TEMPORARY tablespace to use for RQSYS [TEMP]:
Tablespaces summary
PERMANENT tablespace = USERS
TEMPORARY tablespace = TEMP
Installing libraries ......... Pass
Password to use for user RQSYS [ORE1_4RQsys]:
Installing RQSYS data ........ Pass
Installing RQSYS code ........ Pass
Installing ORE packages ...... Pass
Creating ORE script .......... Pass
Installing ORE migration scripts ...... Pass
NOTE: ORE has been enabled for all database users. Next, install the
supporting packages.
You may create an ORE user with the demo_user.sh script, which
automatically grants the required privileges. A complete list of
privileges is available in the script rquser.sql.
To use ORE Embedded R Execution functionality, grant the user
the RQADMIN role.
Please, consult the documentation for more information.
Done
[oracle@oel server]$
Step 3 - Install packages
Download the Download the "supporting" file from OTN. Unzip the file and install the packages.
[oracle@oel ~]# R
Oracle Distribution of R version 3.0.1 (--) -- "Good Sport"
Copyright (C) The R Foundation for Statistical Computing
Platform: x86_64-unknown-linux-gnu (64-bit)
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.
Natural language support but running in an English locale
R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.
Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.
You are using Oracle's distribution of R. Please contact
Oracle Support for any problems you encounter with this
distribution.
> install.packages("/home/oracle/supporting/ROracle_1.1-11_R_x86_64-unknown-linux-gnu.tar.gz", repos=NULL);
Installing package into ‘/usr/lib64/R/library’
(as ‘lib’ is unspecified)
* installing *binary* package ‘ROracle’ ...
* DONE (ROracle)
Making 'packages.html' ... done
> install.packages("/home/oracle/supporting/DBI_0.2-7_R_x86_64-unknown-linux-gnu.tar.gz", repos=NULL);
Installing package into ‘/usr/lib64/R/library’
(as ‘lib’ is unspecified)
* installing *binary* package ‘DBI’ ...
* DONE (DBI)
Making 'packages.html' ... done
> install.packages("/home/oracle/supporting/png_0.1-7_R_x86_64-unknown-linux-gnu.tar.gz", repos=NULL);
Installing package into ‘/usr/lib64/R/library’
(as ‘lib’ is unspecified)
* installing *binary* package ‘png’ ...
* DONE (png)
Making 'packages.html' ... done
> install.packages("/home/oracle/supporting/Cairo_1.5-5_R_x86_64-unknown-linux-gnu.tar.gz", repos=NULL);
Installing package into ‘/usr/lib64/R/library’
(as ‘lib’ is unspecified)
* installing *binary* package ‘Cairo’ ...
* DONE (Cairo)
Making 'packages.html' ... done
>
Step 4 - create a Database User
In this case, I used the demo_user.sh script from the server folder.
[oracle@oel server]$ ./demo_user.sh Oracle R Enterprise 1.4 Server User Creation. Copyright (c) 2012, 2013, Oracle and/or its affiliates. All rights reserved. Checking ORACLE_HOME ......... Pass Checking ORACLE_SID .......... Pass Checking sqlplus ............. Pass Checking ORACLE instance ..... Pass Checking ORE ................. Pass Current configuration ORACLE_HOME = /u01/app/oracle/product/11.2.0.4/dbhome_1 ORACLE_SID = orcl Do you wish to create an ORE user? [yes] Choosing user ORE user to use [rquser]: ./demo_user.sh: line 215: nawk: command not found Password to use for user rquser: Choosing tablespaces PERMANENT tablespace to use [USERS]: TEMPORARY tablespace to use [TEMP]: Tablespaces and user summary PERMANENT tablespace = USERS TEMPORARY tablespace = TEMP ORE user = rquser Creating ORE user ............ Pass Done [oracle@oel server]$
Just in case, provide the RQADMIN grant.
[oracle@oel Desktop]$ sqlplus / as sysdba SQL*Plus: Release 11.2.0.4.0 Production on Wed Dec 9 11:17:54 2015 Copyright (c) 1982, 2013, Oracle. All rights reserved. Connected to: Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production With the Partitioning, OLAP, Data Mining and Real Application Testing options SQL> GRANT RQADMIN to RQUSER; Grant succeeded. SQL>
Step 5 - Test
Setting up the correct library paths and run demo.
[oracle@oel ~]# R
> .libPaths('/u01/app/oracle/product/11.2.0.4/dbhome_1/R/library');
> .libPaths( c(.libPaths(),'/u01/app/oracle/product/11.2.0.4/dbhome_1/lib')
> library(ORE)
Loading required package: OREbase
Attaching package: ‘OREbase’
The following objects are masked from ‘package:base’:
cbind, data.frame, eval, interaction, order, paste, pmax, pmin,
rbind, table
Loading required package: OREembed
Loading required package: OREstats
Loading required package: MASS
Loading required package: OREgraphics
Loading required package: OREeda
Loading required package: OREmodels
Loading required package: OREdm
Loading required package: lattice
Loading required package: OREpredict
Loading required package: ORExml
>
> ore.connect(user="rquser", sid="orcl", host="localhost", password="oracle", port=1521, all=TRUE);
> ore.is.connected()
[1] TRUE
> ore.ls()
character(0)
>
> demo("aggregate", package = "ORE")
demo(aggregate)
---- ~~~~~~~~~
Type to start : return
> #
> # O R A C L E R E N T E R P R I S E S A M P L E L I B R A R Y
> #
> # Name: aggregate.R
> # Description: Demonstrates aggregations
> # See also summary.R
> #
> #
> #
>
Great. Everything works fine. Enjoy.
My first Hadoop application
hadoop com.sun.tools.javac.Main WordCount.java
The next part is the most difficult - you need to create the jar file. I am not a java guru, so I don't really know what should I get at the end, so I've created the jar using the following command:
jar cf wc.jar WordCount*.class
I will come back to the structure of this wc.jar later, so now Jar file is created and we need only to run it, but before that the hdfs directories should be created:
runas /user:hadoop cmd %HADOOP_HOME%\bin\hadoop fs -mkdir -p /test_project/in %HADOOP_HOME%\bin\hadoop fs -put G:\test\gena_test.txt /test_project/in
Everything is ready now, running the application:
hadoop jar G:\test\wc.jar WordCount /test_project/in /test_project/outDoesn't work!!! Getting the following error:
java.lang.RuntimeException: java.lang.ClassNotFoundException: WordCount$TokenizerMapperHm, this is unexpected. Le't google it. At the beginning, I thought that the problem is related with environment variables, therefore I've set all of them (%JAVA_HOME%, %PATH%, %HADOOP_CLASSPATH%) - doesn't help. The next, I've discovered the following warning:
WARN ... No job jar file set. User classes may not be found. See JobConf(Class) or JobConf#setJar(String).
Let's resolve this:
job.setJar("G:\\test\\wc.jar");
//job.setJarByClass(WordCount.class);
But, the error is still shown up. After a couple of hours of useless attempts to resolve this I've found the following topic on stackoverflow.com. The problem was resolved by adding the package into the java class. I tried this and OOOO miracle, the issue is resolved. I think the issue is related with jar file. The jar file I've created previously had two folders :
- test - contains all class files;
- Meta-INF
hadoop jar G:\test\wc.jar test.WordCount /test_project/in /test_project/out
Enjoy.
PS. I've attached the iPhone 6 picture (phone isn't in very good condition I can say :) ). It belongs to one of my associates. The weirdest thing is - it works properly :)
Prince2 Certification
The second week of this year I had spend on Prince2 training. I can say, that the training was intense - every day I had to read ~80 pages and learn a lot of stuff. Additionally, the training included two exams (second and fifth days) :).
Well, today I have received the result and I am glad to say that I'm a qualified Prince2 Practitioner.
dsSize function and TTSize utility in TimesTen
I didn't have an opportunity to write anything lately, because had spent last two months in Nottingham. But now I have a couple days to relax, so I have decided to write about one useful function and one utility in TimesTen. 7.0.5 version is used for the demo (I understand that this is a very old version, but I am a lazy person :) )
As you know, TimesTen stores all information into the memory (RAM) and obviously there are limits for RAM space (nowadays you can buy a server with a couple of TBs of RAM, but still it is not 100 TBs). In this situation, TimesTen users should know how to control the space and how to determine the necessary volume of memory for your system. I have already written one post about the memory allocation here, but now I would like to describe the following:
As you know, TimesTen stores all information into the memory (RAM) and obviously there are limits for RAM space (nowadays you can buy a server with a couple of TBs of RAM, but still it is not 100 TBs). In this situation, TimesTen users should know how to control the space and how to determine the necessary volume of memory for your system. I have already written one post about the memory allocation here, but now I would like to describe the following:
- dsSize function
- ttSize utility
[oracle@localhost info]$ ttisql Copyright (c) 1996-2008, Oracle. All rights reserved. Type ? or "help" for help, type "exit" to quit ttIsql. All commands must end with a semicolon character. Command> connect dslab1; Enter password for 'ttdemo': Connection successful: DSN=dslab1;UID=ttdemo;DataStore=/app/oracle/product/datastore/dslab1;DatabaseCharacterSet=CL8MSWIN1251;ConnectionCharacterSet=US7ASCII;DRIVER=/app/oracle/product/7.0.5/TimesTen/tt70/lib/libtten.so;LogDir=/app/oracle/product/datastore/dslab1/log;PermSize=120;TempSize=50;TypeMode=0; (Default setting AutoCommit=1) Command> Command> dssize; PERM_ALLOCATED_SIZE: 122880 PERM_IN_USE_SIZE: 1222 PERM_IN_USE_HIGH_WATER: 1372 TEMP_ALLOCATED_SIZE: 51200 TEMP_IN_USE_SIZE: 3587 TEMP_IN_USE_HIGH_WATER: 3587 Command>I just created the connection and checked current sizes of the permanent and temporary database partitions. If you add more information in your database, the size will grow. For example, let's create two tables and insert a couple of rows and after that compare results:
[oracle@localhost sql]$ ttisql -f lab1.sql dslab1
Copyright (c) 1996-2008, Oracle. All rights reserved.
Type ? or "help" for help, type "exit" to quit ttIsql.
All commands must end with a semicolon character.
connect "DSN=dslab1";
Enter password for 'ttdemo':
Connection successful: DSN=dslab1;UID=ttdemo;DataStore=/app/oracle/product/datastore/dslab1;DatabaseCharacterSet=CL8MSWIN1251;ConnectionCharacterSet=US7ASCII;DRIVER=/app/oracle/product/7.0.5/TimesTen/tt70/lib/libtten.so;LogDir=/app/oracle/product/datastore/dslab1/log;PermSize=120;TempSize=50;TypeMode=0;
(Default setting AutoCommit=1)
run "lab1.sql";
set autocommit 0;
create table emp (id number not null,
creation date,
name_1 varchar2(40),
name_2 varchar2(40),
surname varchar2(40),
dept_id number,
Primary key (id) );
commit;
create table dept (dept_id number not null,
dept_name varchar2(40),
Primary key (dept_id) );
commit;
insert into dept (dept_id, dept_name) values (1, 'Sales');
1 row inserted.
...
insert into dept (dept_id, dept_name) values (9, 'CEO office');
1 row inserted.
commit;
insert into emp (id, creation, name_1, name_2, surname, dept_id) values (1, sysdate, 'Pety1','Petrovich1', 'Ivanov1',1);
1 row inserted.
…
insert into emp (id, creation, name_1, name_2, surname, dept_id) values (100, sysdate, 'Pety100','Petrovich100', 'Ivanov100',9);
1 row inserted.
commit;
exit;
Disconnecting...
Done.
Command> tables;
...
TTDEMO.DEPT
TTDEMO.EMP
...
Command> dssize;
PERM_ALLOCATED_SIZE: 122880
PERM_IN_USE_SIZE: 1372
PERM_IN_USE_HIGH_WATER: 1372
TEMP_ALLOCATED_SIZE: 51200
TEMP_IN_USE_SIZE: 3650
TEMP_IN_USE_HIGH_WATER: 3713
We see the difference in PERM_IN_USE_SIZE values as expected.
ttSize utility is used to estimates the amount of space that a given table in the data store will consume when it grows to include particular amount of rows. For example, we created the table emp, so let's have a look how we can use ttSize utility:
[oracle@localhost ~]$ ttSize -tbl emp -rows 1000000 dslab1 Enter password for 'ttdemo': Rows = 1000000 Total in-line row bytes = 207078073 Indexes: T-tree index TTDEMO.EMP adds 10541265 bytes Total index bytes = 10541265 Total = 217619338This command shows the approximate amount of memory required for storing 1M rows in emp table.
Hadoop on Windows
A lot of people are interested in Hadoop technology these days and currently I am participating in one project related with this technology. The first task was to install Hadoop - easy, right? But I had been told that we were going to use Windows OS and there is no opportunity to use Linux instead. I conducted a quick research and found the Hortonworks distribution which can work on Windows. BTW, the Hortonworks is the only Hadoop distribution that
works in Windows :). As you see, I didn't have much choice, so I downloaded the software and installation guide.
The first surprise is that Windows Server (2008 and 2012) is supported OS only, but I could manage to find the Windows Server 2008 machine. After that you have to install required software (jdk, python, .net and Visual C++ Distributable Package). No problem. Everything is done. The next step is to run the msi file. I run it and specified all parameters I required. Installation started but suddenly McAfee windows appeared and showed the message that one of the Hadoop files contains a virus and just deleted it. Nice. After that very weird things started happens - I started to get errors like "There is a problem with Windows Installer package. A program run as part of setup didn't finish as expected. Contact the vendor". Hm... I got this error even after I stopped all McAfee services. I am not a Windows expert, therefore I didn't even know what to do :)
After one week struggling with Windows Server 2008 and our firm's security policies I could finally install the Hadoop on Windows. The next post will be about the loading data into Hadoop and Hive.
PS. I would like to thank David Goodhand from Hortonworks support team for his help and very fast response (I didn't expect this to be honest :) ).
The first surprise is that Windows Server (2008 and 2012) is supported OS only, but I could manage to find the Windows Server 2008 machine. After that you have to install required software (jdk, python, .net and Visual C++ Distributable Package). No problem. Everything is done. The next step is to run the msi file. I run it and specified all parameters I required. Installation started but suddenly McAfee windows appeared and showed the message that one of the Hadoop files contains a virus and just deleted it. Nice. After that very weird things started happens - I started to get errors like "There is a problem with Windows Installer package. A program run as part of setup didn't finish as expected. Contact the vendor". Hm... I got this error even after I stopped all McAfee services. I am not a Windows expert, therefore I didn't even know what to do :)
After one week struggling with Windows Server 2008 and our firm's security policies I could finally install the Hadoop on Windows. The next post will be about the loading data into Hadoop and Hive.
PS. I would like to thank David Goodhand from Hortonworks support team for his help and very fast response (I didn't expect this to be honest :) ).
Oracle News
There are a lot of news related with Oracle these days.
The first one is that the founder and CEO of Oracle Larry Ellison decided to leave a CEO job and became a CTO instead. It is kind of a big change in my opinion. Larry has been leading Oracle for the last 37 years and he has archived a great success, therefore now I would expect a new era with Oracle (good or bad will see).
The second one is that OOW conference is started. The first day was about the Cloud, nothing special. Only one announcement was surprising - Oracle Exalytics X4-4 would contain Oracle in-Memory Database! It means that Oracle TimesTen would be replaced by Oracle DB. I think, it is the end of TimesTen! Very sad.
The first one is that the founder and CEO of Oracle Larry Ellison decided to leave a CEO job and became a CTO instead. It is kind of a big change in my opinion. Larry has been leading Oracle for the last 37 years and he has archived a great success, therefore now I would expect a new era with Oracle (good or bad will see).
The second one is that OOW conference is started. The first day was about the Cloud, nothing special. Only one announcement was surprising - Oracle Exalytics X4-4 would contain Oracle in-Memory Database! It means that Oracle TimesTen would be replaced by Oracle DB. I think, it is the end of TimesTen! Very sad.
Подписаться на:
Сообщения (Atom)
