Monday, December 5, 2016

Writing Custom Python Code in ODI

Namaskaram Friends!

Here is a short post on how to write custom python code inside ODI.
What does this mean?

If you look at the concept of Procedures in ODI, you will find that the procedures can be written for a number of technologies, using the technology drop down.
This flexibility makes it a beautiful product, where you can write your procedure in any given technology if the integration project involves a lot of different technologies.

Python is a well known language these days, which is used in a number of industries because of its rich set of libraries.

In this post let's try to explore how to write Python code in ODI ( I am using ODI 11g version here).

Let's work on a very simple scenario:
Lets say we are working on a Data integration project where the client gives us a location to pick the file, and process it using Interfaces (file to table).

The step is,before we process the interface we need to check whether the file exist in the given location/path or not.

To accomplish this task we will write simple python code.

Create a Procedure name it CheckFile, select the Technology as Jython.
              Don't get confused, Jython is Python language implementation that is tightly integrated with the Java Platform. So all the python code is still applicable here

Here is the python code: (take care of the indentation, because python works on the concept of indentation to define its scope, else the code will throw error)

import os
from os import path

def main():
    file_path="D:\\empdata.txt"  
    #The above file_path can also be an odi variable which queries the location from some                     configuration table, that makes the solution even more configurable.
    my_file = os.path.exists(file_path)
    if my_file:
        print("path does  exist")
        return 0
    else:
        raise
#You can write your custom python code here to do mitigations in the else section.
if __name__=="__main__":
    main()



















The procedure checks whether the file exists in the mentioned path or not.

Create a package like this:

When you execute the package, it will first check if the file exist or not, if the file does not exist it will perform the things that you would have mentioned in the else part of your python code.

Hope this would have been useful!!!

Peace

Tuesday, November 29, 2016

ODI Standalone Agents what are they and how to configure

The ODI environment comes with a concept of Agent.

If you have to give a one liner crisp definition about what an ODI Agent is, we will say that it is:
An ODI runtime process that orchestrates ETL flow.

In other words, an ODI agent is a java process, that facilitates the execution of code developed in ODI.

ODI 11g has 2 types of agent: Standalone agent and J2EE agent.
JEE enable ODI agent that allows the ODI agent to inherently take advantage of the application server's enterprise features, such as high availability, connection pooling and so on. However a standalone agent does not give all this features but it does not require a Application server to host it.

In this article, lets try to explore how standalone agents can be configured and used in ODI.

Navigate to the <oracle installed location>\oracledi\agent\bin

The directories may changes from installation to installation , based on where have you installed them.

You will see 2 files there: odiparams.bat and odiparams.sh. The installation folder contains both the files, based on the platform we need to choose which file needs to be edited.

So for a typical windows installation, open the odiparams.bat file. Modify the below parameters.

set ODI_MASTER_DRIVER=oracle.jdbc.OracleDriver
set ODI_MASTER_URL=jdbc:oracle:thin:@localhost:1521:ORCL
set ODI_MASTER_USER=<name of the master repository schema>

set ODI_MASTER_ENCODED_PASS=aYyp5IHh5AGwjgDJw2DgCy

How to generate the encoded password?

In the same folder, type the following command in cmd:

>> encode <master_repo_schema_password>
one encoded string appears.

copy the above encoded string and change the parameter value accordingly.

Similarly change the following parameters:

set ODI_SUPERVISOR=SUPERVISOR

set ODI_SUPERVISOR_ENCODED_PASS=h2yXu8fMrWKoE90kGUN5Extg (generate this string using the encode utility)

One more parameter:
set ODI_SECU_WORK_REP=WORKREP1 (this would be the name of the workrepo associated with the master repo)

Login to ODI studio and Navigate to Topology:

Under Agents: Right Click and select New Agent:
Enter the information:







once the information is entered in the page: navigate to agent/bin folder.
Execute the following command:

>>agent/bin>agent -NAME=<name of the Agent> -PORT=<port no> (The value for the parameters is as per the values given while creating the agent in ODI studio).

The cmd window will display this, once the agent is started:




Go to the agent page in ODI studio and click on the test button, since the agent is running, it will show agent successfully connected message:








That's it, the standalone ODI agent is configured.
Please leave comments if there are any doubts regarding any concepts.

Friday, November 25, 2016

Resetting USER passwords in ODI 11g

Lets try to explore one small thing.

Suppose you installed ODI into your system, now after the installation is done, the user is created.
But due to some other works, you did not login to the ODI studio, or some one messed up with the password etc and some how now, you are not able to access the studio because the password does not work.

The obvious solution for this is to reset the password for the user and start using.

Let's see how this is done.

As you would know, all the users created will go to the tables in master_repository.

Login to master_repository in the database where the repository is hosted.

Execute the following Query:

Select * from SNP_USER where WUSER_NAME= <name of your user>;
You will see one column with the name PASS, now to reset the password we will have to update this column with the new encoded password.

To get the encoded password, we will make use of the encode utility shipped with ODI.

based on the platform/OS where ODI is installed, navigate to the following folder

<ODI installed location>\Oracle_ODI1\oracledi\agent\bin

open cmd/ terminal and change to this directory.

>> cd <ODI installed location>\Oracle_ODI1\oracledi\agent\bin
>> encode <newpassword> (give just the password, arrows are only to show things clearly)

One encoded string will be generated.

Execute the following Query in master repository:

update snp_user
set PASS='aYya6Tx0FiJV,hpw94CKPy' (the above encoded string)
where WUSER_NAME= <name of your user>;

commit the changes.

You are done, navigate to ODI studio and use the new password (the normal string , not the encoded one).

Thanks for reading, see you in another post.