Hadoop on Windows

A lot of people are interested in Hadoop technology these days and currently I am participating in one project related with this technology. The first task was to install Hadoop - easy, right? But I had been told that we were going to use Windows OS and there is no opportunity to use Linux instead. I conducted a quick research and found the Hortonworks distribution which can work on Windows. BTW, the Hortonworks is the only Hadoop distribution that works in Windows :). As you see, I didn't have much choice, so I downloaded the software and installation guide.

The first surprise is that Windows Server (2008 and 2012) is supported OS only, but I could manage to find the Windows Server 2008 machine. After that you have to install required software (jdk, python, .net and Visual C++ Distributable Package). No problem. Everything is done. The next step is to run the msi file. I run it and specified all parameters I required. Installation started but suddenly McAfee windows appeared and showed the message that one of the Hadoop files contains a virus and just deleted it. Nice. After that very weird things started happens - I started to get errors like "There is a problem with Windows Installer package. A program run as part of setup didn't finish as expected. Contact the vendor". Hm... I got this error even after I stopped all McAfee services. I am not a Windows expert, therefore I didn't even know what to do :)

After one week struggling with Windows Server 2008 and our firm's security policies I could finally install the Hadoop on Windows. The next post will be about the loading data into Hadoop and Hive.

PS. I would like to thank David Goodhand from Hortonworks support team for his help and very fast response (I didn't expect this to be honest :) ).




 

Oracle News

There are a lot of news related with Oracle these days.

The first one is that the founder and CEO of Oracle Larry Ellison decided to leave a CEO job and became a CTO instead. It is kind of a big change in my opinion. Larry has been leading Oracle for the last 37 years and he has archived a great success, therefore now I would expect a new era with Oracle (good or bad will see).

The second one is that OOW conference is started. The first day was about the Cloud, nothing special. Only one announcement was surprising - Oracle Exalytics X4-4 would contain Oracle in-Memory Database! It means that Oracle TimesTen would be replaced by Oracle DB. I think, it is the end of TimesTen! Very sad.

Chapter 4 - Key Terminology

This chapter contains only definitions - the most boring chapter I've ever read. Don't know even what to write. Quick read and test is passed (5 out of 6). Anyway, have to learn these definitions :(
Hope the next chapter will be more interesting.

Chapter 3 - Core Concepts (Part 2)

Today I have to understand what Enterprise Continuum (EC) is. First of all, EC is a view of Architecture Repository. EC should explain how the generic solution can be leveraged and managed in order to support the requirements of an particular organisation.

Architecture Repository stores different classes of architectural output at different level of abstraction. AR is consists of the following components:

• Architecture Metamodel
• Architecture Capability
• Architecture Landscape
• Standards Information Base (SIB)
• Reference Library
• Governance Log

Enterprise Architecture Capability (Architecture Capability Framework) - is a set of materials and guidelines. It consists of 7 parts (will not list them here) - OMG I have never seen the word "Architecture", "Capability" so many times on one page :).

Additional thing is enterprise architecture practice must be run like any other operational unit within a business.

The test is done. 6 out of 7 - made only one mistake - forgot that EC is a model for classifying artifacts. Looking forward for the next chapter.

Chapter 3 - Core Concepts (part 1)

I've already written that didn't have much time to read the TOGAF - last Friday was a deadline for the project, therefore I was spending for almost 10-11 hours at work. Finally, it is over, so I can continue the process.

Today is the third chapter. It contains a lot of information and I have to remember all of it. This is not a very easy process, so I decided to split this chapter to two part. This part contains information about ADM (including phases), Deliverables, Artifacts and Building Blocks.

ADM (Architecture Development Method) forms the core of TOGAF and provides the repeatable process for developing architecture. There are 10 phases in ADM.

• Preliminary (Architecture Capability)
• Phase A: Architecture Vision
• Phase B: Business Architecture
• Phase C: Information Systems Architecture (Data and Application)
• Phase D: Technology Architecture
• Phase E: Opportunity and Solutions (forming Target Architecture)
• Phase F: Migration Planning (migration from Base to Target)
• Phase G: Implementation Governance (architecture oversight of implementation)
• Phase H: Architecture Change Management (managing change to the new architecture)
• Requirements Management (examines the process)

Very difficult to remember, spatially G and H phases, but I will try :). The next section describes the structural model for ADM outputs (like requirements, plans etc.). The TOGAF Architecture Content Framework uses three category:

• deliverable (signed off by stakeholders)
• artifacts (aspects of architecture: matrices, catalogs, diagrams)
• building blocks (reusable components)

That is all for today. Tomorrow, will have a look on the rest of chapter 3.

Yesterday - no progress on TOGAF

Unfortunately, could not write about the Chapter 3 yesterday - project deadline is coming, therefore was very busy in the office.

Chapter 2 - Basic Concepts

This chapter contains brief description about TOGAF. The key points are the following:

1. What is TOGAF - The Open Group Architecture Framework. TOGAF is a tool. The key to TOGAF is the method - ADM (Architecture Development Method).
2. Structure of TOGAF document (7 points):
• Introduction
• ADM
• ADM Guidelines and Techniques
• Architecture Content Framework (ABBs)
• Enterprise Continuum and Tools
• TOGAF Reference Models (TFM and III RM)
• Architecture Capability Framework
3. What is Enterprise, Architecture, Why do you need Enterprise Architecture (a lot of reasons) 
4. Architecture Domains Supported by TOGAF
• Business
• Data
• Application
• Technology

Well, the test at the end was not very difficult, so I got 100% again, but the topic is quite unusual for me, therefore I need to sleep on the info I've read from this chapter (maybe will read it again).

Chapter 1 - Introduction

The first chapter of the book consists of general information about TOGAF certification: certification levels (this I already covered in the previous post), TOGAF certification principles (Openness, Fairness, Market Relevance, Learning Support, Quality, Best Practice).

Additionally, there is a lot of information about the exam. The exam consists of 40 questions and you have 60 minutes to go though all of them. Each question has only one right answer. To pass the OG0-091 test you should answer for at least 55% of all questions. It means that you should answer correctly for at least 22 questions out of 40. The exam cost is $320. The majority of the questions are about ADM phases.

So, this chapter was quite easy to read, because it doesn't contain any difficult information (but I have to remember the list of all topic areas). Test is passed on 100%. Looking forward for the second chapter.  

NEW CHALLENGE - TOGAF

Have not written a lot since I found new job - new people, new environment, new business etc., plus I am a very lazy person :), but now I have decided to struggle with my inactivity and start a new challenge - take two exams for becoming a TOGAF certified member (about TOGAF certification you can read here ) . This should help me with my carrier (not sure about it to be honest :) ). So let's start.

TOGAF certification consists of two levels:

• TOGAF Foundation (level 1)
• TOGAF Certified (level 2)

The highest level is the TOGAF Certification. There are two ways to get this certification:

• TOGAF Part 1 exam (OG0-091) +  TOGAF Part 2 exam (OG0-092)   
• TOGAF 9 Combined Part 1 and 2 exam ((OG0-093 - for advanced people)

I am not an advanced person, therefore I am following by the first option. Start from TOGAF Part 1 exam. There are a lot of documents about this exam, so I just downloaded one preparation book. This book contains 13 chapters and I am going to read one chapter each day (max 2 days) and write about it in this blog - this will help me with exam preparation and additionally, help me to do something at all :)

"Let the Seventy-forth Hunger Games begin" - quotes from "Hunger games" book.

Oracle TimesTen 11.2.2.7.0 is released

Oracle introduced 11.2.2.7.0 version ( available on OTN ).

The key new feature in this release in my opinion is the following:

"In this release, checkpoint files can be read in parallel. Checkpoint reads use the value of the RecoveryThreads connection attribute when that value is greater than one. The maximum number of threads used for parallel checkpoint read is eight, regardless of the value of RecoveryThreads. Parallel checkpoint reads can improve load time when reading from Solid State Disks (SSD)." This could improve the load time for big databases, but to be honest, there are no major changes in this release. Still waiting the 12C version.

All new features you can find here.

Oracle PL/SQL code execution from Timesten

As you know, TimesTen has opportunity to execute SQL statements directly from Oracle DB (using Passthrough). What about PL/SQL? Unfortunately, documentation says:

"A PL/SQL block cannot be passed through to the Oracle database for execution.
Also, you cannot pass through to Oracle for execution a reference to a stored procedure or function that is
defined in the Oracle database but not in the TimesTen database."

It is true, you can not use passthrough feature for PL/SQL, but you can use it for SQL. It means that you can execute PL/SQL from Oracle DB which could be executed from SQL. For example:

Suppose, we have a function in Oracle which we would like to execute:

SQL> create function test_r return number
is
begin
  return 1;
end;
/

Function created.

SQL> select test_r from dual;

    TEST_R
----------
         1

SQL>

Create one table which doesn't exist in TimesTen side.

SQL> create table tt (id number);

Table created.

SQL> select * from tt;

no rows selected

SQL> insert into tt values (3);

1 row created.

SQL> commit;

Commit complete.

SQL>

In TimesTen:

 
Command> select test_r from dual;
 2211: Referenced column TEST_R not found
The command failed.
Command> 
Command> set autocommit 0;
Command> call ttOptSetFlag('PassThrough', 1);
Command> select * from oratt.tt;
< 3 >
1 row found.
Command> select oratt.test_r from oratt.tt;
< 1 >
1 row found.

This method, of course, has some restrictions for PL/SQL execution, but in any case, it is better than nothing :)