How can I migrate XML-based C1 site to the SQL Server-based site?

Topics: General, Standard packages
Oct 13, 2010 at 9:16 AM

i see composite C1 store data in xml file. How can I migrate XML-based C1 site to the SQL Server-based site?

Oct 14, 2010 at 1:06 AM
Edited Oct 14, 2010 at 1:09 AM

Composite C1 come with a 'XML data provider' but you can migrate all data from XML to SQL - the fast way is to visit http://www.composite.net/C1/Offerings/SupportandServices/Developer-Admin-Tools.aspx and purchase the "MS SQL Server Connector". This will give you a 'SQL Data Provider' and a automated migration tool which enable you to switch from XML to SQL in a matter of minutes, if you have a sql database and dbo login available.

All your data types (existing C1 data types and types defines by you or via packages) and their data will be migrated to SQL. Each data type will become a nice table with a properly typed column for each data type property (the tables look good) - so you can do std. SQL directly on it if that need should ever arise.

Since all data query logic is captured in LINQ you won’t have to rewrite any code (well, if you have written fairly complex LINQ constructs that 'LINQ for SQL' does not support, you need to rewrite that code) so the migration process should be fairly painless and you can do it when needed (you don't have to decide early on).

Please note that Composite C1 can perform quite well when running on XML - unless you have a lot of write operations or huge amounts of data. The XML data provider keeps data in mem and write the entire xml document to disk on changes, so it’s super fast for reading but can consume mem and does not scale well in terms of updating data.

Oct 14, 2010 at 12:58 PM

What about this solution ?

Migrate a C1 website from one data store type to another

http://docs.composite.net/C1/Packages/CompositeC1Tools/CompositeToolsDataStoreMigrator.aspx

Oct 14, 2010 at 2:56 PM

That tool is actually part of the commercial offering I described above (i.e. not free).

We will update our package descriptions so it is easy to identify if a package is free or not (most are, but we have about 4 "purchase only" packages).

Oct 15, 2010 at 4:19 AM

why don't use sql server to store data. i know before your team open source composite c1, this source can choose sql server or xml to install. i find way to install to sql server.

Oct 15, 2010 at 7:45 AM

I can only agree, than SQL server is a must.

Take a look at the other Open Source CMS systems, there are on the net. They all have SQL as an option, sure we can buy the commercial package, but it would be great if it was a free option.

Oct 15, 2010 at 8:20 PM

Having started playing around with Composite I'd have to say that I'm impressed. Having said that, here's my vote for the SQL server functionality, open source please... This is too basic of a need for any kind of more or less functional site. If you don't release the data adapter into open source, sooner or later someone will write there own... and release it  for free. Anyway, good job to the Composite team.

Oct 16, 2010 at 3:24 AM

download Composite C1 for VS. In this package have one dll name composite.mssqlserver. Add new line at composite.config after line 36:

<Composite.Data.Plugins.DataProviderConfiguration defaultDynamicTypeDataProviderName="DynamicSQLDataProvider"> 
  <DataProviderPlugins>    
	<add connectionString="Data Source=MACHINENAME;Initial Catalog=DATABASENAME;Integrated Security=True" sqlQueryLoggingEnabled="false" sqlQueryLoggingIncludeStack="false" type="Composite.MSSqlServer.SqlDataProvider, Composite.MSSqlServer" name="DynamicSqlDataProvider">
		<LoggingIgnoreInterfaces>
			<add interfaceType="Composite.Data.Types.IUserConsoleInformation" />
		</LoggingIgnoreInterfaces>
	</add>
  </DataProviderPlugins>  </Composite.Data.Plugins.DataProviderConfiguration>

After config, install composite c1 will store data in sql server. Hihi.

 

Oct 18, 2010 at 4:48 AM

As far as the config goes - it's correct. You still need a data provider though, like Composite.Plugins.Data.DataProviders.SqlDataProvider, which is not part of the open source. So unless you buy it or write your own - you're out of luck.

Oct 18, 2010 at 8:42 PM

 

@hongdiepkien I do love a good hacker ;-) I hope you liked the tables you ended up with.

@rakol and @xsejer - I'd like to keep this topic open for discussion, let me briefly describe our motives and then throw out some ideas.

We choose to make the Sql Provider a commercial extension for the following reasons:

  1. We need to make some money to fund the development - I guess it is not a shame to admit that.
  2. We believe that you can get pretty darn far on the Xml Provider
    (75% our our own sites run on Xml + these sample sites; www.simcorp.com and www.unipension.dk (same site contains www.arkitektpension.dk www.pjdpension.dk www.mppension.dk + minor english versions).
  3. When you run on Microsoft SQL Server, chances are you are already paying money for it (to Microsoft), so "free" must be less important.
  4. If you end up with a website where size, complexity or features require you use the SQL Provider, chances are there is a customer paying for all that development
  5. You are not "stuck" with Xml - if the need really arise you can move to SQL
  6. We would like to promote the Xml Provider because it's "all files and data in one directory structure" makes Composite C1 development, deployment, backup much easier (xcopy)

I'd like to dig into the background for your requests to open source the SQL Provider so I better understand where you are comming from - which of the following would you say apply to your objection?

a) you believe it's amateurish to build websites with flat XML files as a physical store - you would never dream of doing it.
b) you want the security of transactions and the ability to access data using the great tools that exists for SQL Server
c) you would prefer to programatically read/manipulate data more directly using SQL (rather than using the DataConnection in Composite C1)
d) you feel that the SQL Provider should be open source because you want the ability to maintain it (as a matter op principle you want your software free, as in freedom)

Question is if we are talking price, taste or basic principle - and question is what model could make everyone happy? In short, we do not want to make money on developers, but rather on companies that want "something extra". We could imagine some model where the SQL Provider is freely available for evaluation, non-profit and community use. We could also imagine seriously rethinking the price.

Opinions?

Oct 22, 2010 at 1:22 PM

I think its fine that you can buy the MS SQL provider. To license it as free for development and non-profit organizations would really be showing your commitment :) 

You should make it Open Source anyway though, even if it still was to cost money. This way you can get people to find bugs and optimize it, or release providers for MySQL, VistaDB, SQLLite, Oracle and other database. Without knowing the inner details i can't imagine it being rocket science.

Oct 30, 2010 at 1:11 AM

@mawtex - thanks for following up on this topic.

Here are my thoughts on your comment:

  • I'm sure you've heard of SQL Express. Free, powerful, rather large limit level... I would argue that most smaller sites out there run on SQL Express, which is more than enough for them.
  • Real database is not only for big boys these days. Most web applications, including open-source ones, use them pretty much as standard out of the box.
  • Without thinking that XML store is amateurish, and with all my respect for XML, I think we all agree that XML will hit its performance limit way before SQL Server (even Express version). Makes a database a much more versatile solution.
  • Data integrity. I simply don't have the same level of trust towards an XML store in this aspect as I do towards a database.
  • Database tools. Can't match those with XML. XCopy vs DB backup? Frankly I would prefer the later.

While I do understand that you need to build a working business model around your now open-source product and some functionality should be targeted towards paying business customers, I simply don't think that the SQL Provider is one of them. I guess a much better model would be requiring a license only when used to access the full SQL Server (Standard, Enterprise, etc.)

Nov 3, 2010 at 10:35 PM

Regards data integrity, speaking of the devil... i just had to do clean install of my installation after my VMWare crashed and apparently left some xml-files in corrupt state. I couldn't even open up the Console, because of an exception already in Composite Application Initialization. Some error about an xml file missing its root.

So, yeah... just SQL Lite or Vistadb would be fine as a file based alternative to xml files.

Dec 18, 2010 at 2:29 AM
hongdiepkien wrote:

i see composite C1 store data in xml file. How can I migrate XML-based C1 site to the SQL Server-based site?

Please check update at SQL Server support will become available free of charge.