YetAnotherForum
Welcome Guest Search | Active Topics | Log In | Register

Backfiling before creation date Options · View
jiteshraniga
#1 Posted : Wednesday, September 14, 2011 12:42:54 AM
Rank: Member
Groups: Member

Joined: 6/9/2011
Posts: 12
Location: Brisbane
Hi all,

With the help of a bit of searching the forums I put together some code for manual entry of some tags via excel. I was entering some values today when I encountered a problem. I had to create a new tag for this data which I did today but the data was dated for last month and would not write to the tag. Is there a way of getting around this problem.
I have temporarily changed an old tag which was neever used and placed the data there but for future instances (likely to happen again soon with a bunch of tags) what would be the best way to tackle this problem. I have created a handful of placeholder tags for now but I don't like this solution very much.

Any help is appreciated.

Regards,
Jitesh
Sponsor  
 

OSIsoft vCampus is a subscription-based, online offering that consists of providing everything people need to develop applications on the PI System.
We invite you to take a "tour" of the OSIsoft Virtual Campus - also feel free to consult the FAQ  or contact OSIsoft vCampus for more details.
jhcoxx
#2 Posted : Wednesday, September 14, 2011 4:09:38 AM
Rank: Member
Groups: Member

Joined: 7/5/2010
Posts: 27
Location: Texas
It's been a long time since I had to do this, but what worked was to write at least one value after (more recent than) the Pt Created value, then delete the Pt Created value. After that, 'retrospective' values could be written back before the old Pt Created value's timestamp.

This was so long ago that we were still heavy into PI-API so that's what was used to do it.

It's entirely possible there is a more elegant way to do this with the PI-SDK, or for that matter, PI-SMT. Maybe you'll get luck and someone else can give you better advice!

jhcoxx
squatty
#3 Posted : Wednesday, September 14, 2011 7:45:21 AM
Rank: Advanced Member

Groups: Member

Joined: 4/7/2011
Posts: 137
Location: KZN, South Africa
The route you would follow depends on a couple of things.

1. How many tags do you plan to backfill?
2. The amount of data being inserted into the archive for backfilling will depend the route you will follow
2a)For a couple of tags with small amounts of data you can follow jhcoxx suggestion
2b)For large amount I would suggest inserting these into a csv file and using piconfig on the PI server to backfill the data
3)Backfilling with and without compression?
4)Always create some tags for backfilling, we created 1000 tags just for this purpose, and 10 years down the line I'm very happy we did.
caffreys_col
#4 Posted : Wednesday, September 14, 2011 10:07:35 AM
Rank: Advanced Member
Groups: Member

Joined: 4/6/2011
Posts: 40
Location: Grimsby UK
To follow on from squatty's suggestion also create a set of peformance equations that are blank or just hold a value of 1. When you want to do some PE calcs on historical PI data but need a performance equation to back fill the data, then just edit one of backfill PE's and recalculate it using the PI recalculator function. It will then recalculate the data as far back as the PE was created if necessary.

Below is the OSI tech support page on how to backfill data. You might need to refer to the manuals for some bits tho!

Issue
How do you backfill existing PI 3 archives with data from new tags? What is the procedure for backfilling new data into old archives?
Solution
The following steps provide a general outline of how to do this. The PI System Manager Resources pages also have information on backfilling data.

1. Add the new tags to the existing PI Server archive using you preferred method (Tag Configurator is the easiest way to batch load a lot of tags at once, or you can use the PI SMT Point Builder plug-in to create them one at a time).

2. Use the Offline Archive Utility to reprocess all archives that you wish to add data to for these new tags. Please see the PI Server System Management Guide chapter on Using the Offline Archive Utility (piarchss) for information on how to use this utility or consult KB KB00159 for additional information.

3. Backfill the data.

Please note that the above procedure does not make use of compression so all values that are backfilled will go into the archive, regardless of compression settings. If you wish to implement compression on your data, you will need to add the following steps between steps 2 and 3 above:

A. Disable shutdown events for each of the new tags. Usually this is accomplished by setting the Shutdown attribute to 0 for each tag.

B. For each of the new tags that you wish to backfill, add one value with a timestamp before or at the start time of the backfill period.

C. Delete the snapshot value for each of the tags from step A.

Once these steps are complete, you can move on to backfilling your data, and compression settings will take effect. Please note that Steps A-C can only be implemented on PI 3.3 servers or later, since it is not possible to delete snapshot values in versions of PI prior to 3.3.

It is also important to note that the above procedure could result in the target archive filling before all data is backfilled. Therefore, it is generally considered a good idea to use dynamic archives for backfilling. If desired, you can easily reprocess them into fixed archives once the backfilling is complete. If you don't want to use dynamic archives, it is possible to experiment and find out how much space is required per time period, then use that period when creating fixed archives. One way to find out this info is to have the backfilling program work hour by hour for all points and then use piartool -al to observe filling up rates. PI 3.4 will expose performance counters that will allow you to monitor the available space in each archive over time, which could also help.
Notes
RJK Solutions
#5 Posted : Friday, October 07, 2011 6:06:39 PM
Rank: Administration

Groups: Administration

Joined: 6/20/2008
Posts: 617
Location: Cheshire, United Kingdom.
I would certainly recommend the official OSIsoft approach to this. Simple put, create the tags you need then reprocess the archives that cover the time period you need to backfill. Now, if you need to backfill data for the last 10 years then the data must be important enough to go through the administration overhead of using piarchss on lots of archive files. Plus, a reprocessed archive switches the optimisation from time based to point based (by default), which can bring performance improvements.
Principal Consultant
Real-Time Data Management @ Wipro Technologies
jiteshraniga
#6 Posted : Friday, October 14, 2011 2:58:19 AM
Rank: Member
Groups: Member

Joined: 6/9/2011
Posts: 12
Location: Brisbane
Thanks for all your help. It should only be a few tags that need backfilling (10 at most). I was aware of the Performance equation backfilling issue and had 100 equations created before. Didn't think about regular tags. There will be no compression applied to the tags. I will give some of the solutions a go when I have some time.

Thanks
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.