Customers often ask us whether archiving or partitioning is the best practice for ILM. This question usually is based on a misunderstanding, because both can be used to address performance and storage cost issues. Partitioning and archiving is not an either/or choice. Rather they are complimentary, as evidenced by this from Gartner. So the correct answer to this question is yes, and Solix EDMS supports partitioning, archiving, and retiring data to the cloud for all your applications across your enterprise.

According to Orafaq, database partitioning is an option that database companies sell on top of a database license . Partitioning allows DBAs to split large tables into more manageable "sub-tables", called partitions, to improve database performance, manageability, and availability. It supports scaling of certain applications and provides the ability to leverage tiered storage to take advantage of lower-cost storage options. Whether partitioning is a good data management strategy depends in large part on how the individual application is architected. For instance, d ata warehouse applications are generally good partitioning candidates because date is usually a key dimension and can be used to spread the data out evenly (by years, quarters, months, etc.) This segregation can be used to improve query performance by excluding ‘offline’ partitions that are no longer needed for day-to-day operations. If applications are not architected with partition elimination strategy in their queries  it doesn’t significantly improve database performance as partitioned data still resides in the database as against moving complete sets of data out of the database does.

Database archiving, on the other hand, is the process of moving data that is no longer actively used to a separate data storage device such as SATA disk or tape, for long-term retention. This strategy is recommended for older data no longer active in the production environment but important for reference or regulatory compliance. Generally, this is achieved by creating two versions of each table, one for active data, and one for archived data. However, this means that all queries that need to access the archived data must now select a union of the active and archive tables. Database archiving products manage this data movement in an optimized manner while maintaining the integrity of the application.

Oracle, in its white paper “ Using Database Partitioning with Oracle E-Business Suite", for instance, recommends partitioning for improving performance and saving cost and reducing maintenance activities by moving old partitions to read-only table spaces (pg 49). The biggest problem when defining partition strategies for the Oracle E-Business Suite is that the application code has been prebuilt, which limits the choice of keys. Furthermore, changes to the partitioning strategies delivered with the base product are not recommended or supported. Therefore, the introduction of partitioning is complex, requiring both substantial analysis and robust testing of its effect on different components in your workload. This makes archiving of older data more attractive. A lso, by itself database partitioning does not meet long-term archiving and compliance requirements. On the other hand, transaction-based archiving alone may not be able to keep up with the volume demands of certain applications. Together they can provide a highly complementary solution that maintains performance of the production database while saving money.

Solix EDMS is designed to help with your partitioning and archiving strategy, as Gartner acknowledges in reports such as “ Solid Vendor Solutions Bolster Database Archiving Market ” by Sheila Childs and Alan Dayley. In that report, Solix is listed along the four important vendors for database archiving. Solix EDMS can be used to apply a consistent set of company ILM policies for data archiving and data retirement across all your data and applications. And it can provide a solution for retaining read-only access to data when retiring obsolescent applications, an important strategy for data center efficiency that is strongly supported in the Gartner report cited above.

On another note, I am happy to announce that we will co-sponsor www.ilmsolutions.net . This is an excellent resource for end-users researching ILM strategy, as it publishes all the information on the market, including industry white papers, analyst reports and customer case studies.

I have great news this month. First, John Ottman, one of the pioneers of the ASP/Cloud industry, has joined Solix. I have known John for some time, back from his Corio days and then as President of Princeton Softech (acquired by IBM).

Second, we just announced Solix Cloud, providing automated on-demand data archiving or applications retirement over the network for enterprise customers. I had been trying to recruit John for some time, and one fine day he said, if we can convert Solix into a cloud company, he would be happy to consider. That seemed like a great opportunity and here we are.

Archiving vs backup

I can best illustrate that with a true story. I won’t name the company, but let’s say they are a fairly large corporation. They are a large Oracle customer, using Oracle’s ERP suite for the last 20 years. They are experiencing huge data growth and having performance issues and called us. For the past two decades each time they upgraded they first made a backup, which they called their archive, and then re-implemented the solution. As a result, they now have more than five application versions. When they need to find a transaction, they have to choose the right version to do the search. Considering the number of application instances they are running, application maintenance costs, potential software licensing issues, and more importantly the lack of business intelligence across their last 20 years of operations, the total cost is mind boggling.

That brings me to the point; many folks are still confused about the differences between archiving and backup. According to Wikipedia definition – a backup or the process of backing up is making copies of (active)  data  which may be used to restore the original after a  data loss . A rchiving is the process of retaining aging (semi-active or inactive) data for purposes other than recovery, such as reducing storage costs or shrinking the backup window. It is often used for records management and compliance to government and industry regulations.

Now the question: Why not use backup as an archive. That’s never a good idea, according to Gartner (“Does Integrated Backup and Archiving Make Sense” by Analysts David Russel and Sheila Childs, pub. 21 March 2012). Basically companies should backup active data that changes over time and that the company will need to restore as quickly as possible to support ongoing business operations if the primary database is damaged or inaccessible for some reason ; or some portion of that data is corrupted or lost to a hardware failure. Inactive data, that is no longer changing or being accessed, should be removed from both the primary database and the nightly backups both to save money, speed normal operations, and decrease backup and recovery times. This data should be archived both for compliance and to preserve historical information about the company as well as for long-term trend analysis.

Archiving/ILM products can effectively identify data that needs to be preserved based on any number of criteria or polices. When data is archived, it is generally moved with context while maintaining application or referential integrity, so that when and if it becomes necessary to find that data in the future, it is relatively easy to do so. Archiving and backup, therefore, are inter-twined as part of a comprehensive data life cycle management strategy, according to Gartner.

Solix Cloud

So why is Solix delivering the only cloud-based data archiving and retirement service? Archiving is a best practice due to its positive impacts on application performance, DR strategies, application upgrades, backup times, and infrastructure costs. The technology for business-rules-based automated archiving  as exemplified by the Solix archiving tool set, is mature. Yet, considering the overall market size, it is still at its infancy with a large number of companies still not implementing archiving despite the advantages it delivers, in part for financial reasons.

So after rigorous work, I am happy to say we put together the Solix C loud offering, providing both archiving and retirement function for enterprise customers. Eureka! We are the only company to offer this. For companies considering archiving but constrained by budget or resources , this is the solution. This might just be what the market needs to expand. Further, this is perhaps a great first step for enterprises considering moving some operations to the cloud. These are exciting times, more to come…

As CPU power continues to grow according to Moore’s Law and increasing numbers of customers see the advantages of near real-time data access, big data ingestion and predictive analysis and similar advanced computing strategies, data read/write speeds have become a major barrier to achieving high performance. The basic problem is that disk spins at the same speed that it did 30 years ago, which means disk reads and writes, particularly random accesses, can only happen so fast. And the steady increase in disk capacity has, if anything exacerbated rather than ameliorated this problem.

In the last two years, however, a new technological answer has entered the market – NAND flash storage providing a 20X to 40X performance improvement on PCIe cards plugged directly into the server. While initial NAND flash implementations were very expensive and exotic, their price has been falling steadily due mainly to the heavy demand from the consumer marketplace. Flash SSDs are highly reliable and have much lower power consumption and therefore cooling costs than disk storage, in part because they only need power when they are actually in use. For these reasons, NAND Flash technology is becoming a disruptive force in the storage hardware industry. Several vendors are offering flash in different configurations at different price points, and flash storage has become a practical and much higher performance, alternative to high-speed disk for data-intensive, business-critical applications such as ERP and CRM systems even for SMBs.

NAND flash, however, is significantly more expensive than SATA disks on a per Gbyte basis. Hence, the optimal way of using flash is to replace high-speed disk entirely for the most frequently accessed Tier 0 and Tier 1 data in business-critical systems, where the important cost measure is cost per access, while historical data that is not often used should be migrated to high-density SATA disk systems. The concept of flash architectures is that flash will only be a small fraction of the total storage on the system and that it will hold only the fraction of data that is used intensively – for instance active purchase orders data from ERP systems.

This creates a challenge for many businesses, including both large enterprises and SMBs that have never established an effective data tiering system. The ERP databases of many companies contain years of historical data, much of it seldom used but retained for tax, compliance, and decision-support uses. This data slows operations at every turn, creates problems running backups and restores, and takes up space on expensive storage systems. Therefore, before companies can move to flash storage, they need to build efficient tiered data storage architecture first. This means that they need an information lifetime management (ILM) system that can automatically tier data according to the level of demand and/or business rules and archive the older historical data in a way that minimizes space while making it available when needed.

You need a vendor like Solix with the vision to understand the disruptive forces now just starting to impact data storage, including flash, and that is developing new versions of their product sets that will work with the new IT architectures that are evolving today. These are exciting times for data technology. After a decade of essentially quiet slow development, it is being rocked by new physical technologies as well as whole new kinds of database technology such as Hadooop and a tremendous influx of data. Today these sound like exotic technologies, but soon your business’s very survival in a highly competitive marketplace may depend on their successful adoption. It is not too soon to be thinking about them and preparing, and ILM technology will play a vital role. You will reap benefits from increased efficiency, higher service levels, and shorter backup and restore times, immediately.

We had a good year in 2011. We won some significant new customers, signed up new resellers worldwide, made great strides in improving and adding to our product set, and our win-rate continued to increase. We are looking forward to an even better 2012, with upcoming new product announcements and new partnerships. We are excited about the future and wish all our customers and partners a great year ahead.

Solix Technolo gies today is the only independent pure play information lifecycle management tools vendor in the market, and indeed has been so for several years. Our competition is not from other entrepreneurial software vendors but from some of the largest companies in the industry – IBM, Hewlett-Packard & Informatica – each many times our size in overall resources. On paper at least we are the sole David facing three Goliaths.

Yet despite this disparity in overall size, Solix has more than 150 customers who use our technology to manage their data and that is the heart of their organizations. And we are advancing into the era of Big Data and Cloud and new tools to meet new challenges.

So how do we manage this? Recently the editor of  ceo2ceos.com asked me exactly that question – how can we continue to compete against these giants, and what is my advice for aspiring entrepreneurs who want to use new ideas and approaches to take on established players?

The interesting fact about our competition is that they do not attempt to compete against us based on technical superiority. Rather they cite their greater size and spread fear, uncertainty, and doubt (FUD) about our viability. The fact is Solix has been in business since 2002 and has weathered tough economic times, which economists rate as the worst financial disaster to befall the United States since September 1929. Through it all, while other companies cut back, we maintained our staff, customer service level, and development schedule. Today Solix remains financially stable, with a bright future ahead.

To answer the editor’s first question, I said that this continued success is based on three legs:

1. Engineering Focus – We maintain best-of-breed status in the industry with a constant focus on and investment in building a great technology. Our constant goal is to be the technology leader in ILM.

2. Marketing by our customers – Our answer to our competition is let our customers speak for us. We have many happy customers such as Honeywell, Duke Energy, American Tires, who are happy to participate in joint webinars and customer case studies. We further closely work with them on the latest developments and direction and to hear what they need. We use that feedback for our future product direction.

3. Partnerships – We are continuing to invest on our relationship with our partners, including our OEM relationship with Oracle Financial Services, and we have more such partnerships in the works that will help us expand our presence in the market.

This strategy works. My advice to other entrepreneurs is to dream big and invest in realizing those dreams in terms of developing innovative, superior technology and strong communications with their customers and prospects. Listen to what they need and let that guide your development direction.

We wish our customers, partners and employees wonderful Holidays.

Solix Technolo gies today is the only independent pure play information lifecycle management tools vendor in the market, and indeed has been so for several years. Our competition is not from other entrepreneurial software vendors but from some of the largest companies in the industry – IBM, Hewlett-Packard & Informatica – each many times our size in overall resources. On paper at least we are the sole David facing three Goliaths.

Yet despite this disparity in overall size, Solix has more than 150 customers who use our technology to manage their data and that is the heart of their organizations. And we are advancing into the era of Big Data and Cloud and new tools to meet new challenges.

So how do we manage this? Recently the editor of  ceo2ceos.com asked me exactly that question – how can we continue to compete against these giants, and what is my advice for aspiring entrepreneurs who want to use new ideas and approaches to take on established players?

The interesting fact about our competition is that they do not attempt to compete against us based on technical superiority. Rather they cite their greater size and spread fear, uncertainty, and doubt (FUD) about our viability. The fact is Solix has been in business since 2002 and has weathered tough economic times, which economists rate as the worst financial disaster to befall the United States since September 1929. Through it all, while other companies cut back, we maintained our staff, customer service level, and development schedule. Today Solix remains financially stable, with a bright future ahead.

To answer the editor’s first question, I said that this continued success is based on three legs:

1. Engineering Focus – We maintain best-of-breed status in the industry with a constant focus on and investment in building a great technology. Our constant goal is to be the technology leader in ILM.

2. Marketing by our customers – Our answer to our competition is let our customers speak for us. We have many happy customers such as Honeywell, Duke Energy, American Tires, who are happy to participate in joint webinars and customer case studies. We further closely work with them on the latest developments and direction and to hear what they need. We use that feedback for our future product direction.

3. Partnerships – We are continuing to invest on our relationship with our partners, including our OEM relationship with Oracle Financial Services, and we have more such partnerships in the works that will help us expand our presence in the market.

This strategy works. My advice to other entrepreneurs is to dream big and invest in realizing those dreams in terms of developing innovative, superior technology and strong communications with their customers and prospects. Listen to what they need and let that guide your development direction.

We wish our customers, partners and employees wonderful Holidays.

Solix Technolo gies today is the only independent pure play information lifecycle management tools vendor in the market, and indeed has been so for several years. Our competition is not from other entrepreneurial software vendors but from some of the largest companies in the industry – IBM, Hewlett-Packard & Informatica – each many times our size in overall resources. On paper at least we are the sole David facing three Goliaths.

Yet despite this disparity in overall size, Solix has more than 150 customers who use our technology to manage their data and that is the heart of their organizations. And we are advancing into the era of Big Data and Cloud and new tools to meet new challenges.

So how do we manage this? Recently the editor of  ceo2ceos.com asked me exactly that question – how can we continue to compete against these giants, and what is my advice for aspiring entrepreneurs who want to use new ideas and approaches to take on established players?

The interesting fact about our competition is that they do not attempt to compete against us based on technical superiority. Rather they cite their greater size and spread fear, uncertainty, and doubt (FUD) about our viability. The fact is Solix has been in business since 2002 and has weathered tough economic times, which economists rate as the worst financial disaster to befall the United States since September 1929. Through it all, while other companies cut back, we maintained our staff, customer service level, and development schedule. Today Solix remains financially stable, with a bright future ahead.

To answer the editor’s first question, I said that this continued success is based on three legs:

1. Engineering Focus – We maintain best-of-breed status in the industry with a constant focus on and investment in building a great technology. Our constant goal is to be the technology leader in ILM.

2. Marketing by our customers – Our answer to our competition is let our customers speak for us. We have many happy customers such as Honeywell, Duke Energy, American Tires, who are happy to participate in joint webinars and customer case studies. We further closely work with them on the latest developments and direction and to hear what they need. We use that feedback for our future product direction.

3. Partnerships – We are continuing to invest on our relationship with our partners, including our OEM relationship with Oracle Financial Services, and we have more such partnerships in the works that will help us expand our presence in the market.

This strategy works. My advice to other entrepreneurs is to dream big and invest in realizing those dreams in terms of developing innovative, superior technology and strong communications with their customers and prospects. Listen to what they need and let that guide your development direction.

We wish our customers, partners and employees wonderful Holidays.