Scipio setup for many organisations


#1

Hello,
I have been really looking at this tool lately. Can someone please explain or point me to how the setup of multiple organisations can be done.
From secondary apps --> Setup: You can create many companies, departments stores and warehouses. Question is Can a company with independent structures run on one instance? for example if I create many organisations in setup, how is the data managed, is it separated or mixed? Can many organisations or departments be operating the software at the same time without data interferance?

Thanks.


#2

The truth is the “setup” app is not very useful or practical. If you want a serious deployment of ofbiz, you’ll need much more than do a few basic steps. Not only that, you’re going to have to be able to repeat the seeding process. The only way to do this is by creating SEED DATA for your company, and have the ability to load it over and over again until it works. Regarding your multi-company question, I’m sure it can be done (others can chime in).

Here is the gist of it:

  1. Clear the DB (drop/re-create as necessary)
  2. Load seed,seed-initial
  3. CREATE/EDIT SEED data for your basic company
  4. LOAD seed data for your basic company

[repeat #3, #4 until you get it right]

This is the hard, laborious part, because you’re going to be editing, loading, editing, loading, over and over until it looks right. Every step of editing is NOT via the GUI, but via editing the seed data.

Here are the steps:

  1. Create a hot-deploy component called “company-data”
  2. Create two ofbiz-component.xml files, one in hot-deploy, and one in company-data
  3. The first ofbiz-component.xml (in hot-deploy), loads “company-data”
  4. The second ofbiz-component.xml (in hot-deploy/company-data), loads the XML files (in the data directory)
  5. Load up your data directory (hot-deploy/company-data/data/ with a separate XML file(s) for each item listed in the ofbiz-component.xml .

How to you create the initial seed data? From the demo data (of course). If you load the demo store, these are the files, IN ORDER, that are loaded.

/opt/scipio-erp/framework/security/data/PasswordSecurityData.xml
/opt/scipio-erp/framework/security/data/SecurityGroupDemoData.xml
/opt/scipio-erp/framework/common/data/CommonSecurityGroupDemoData.xml
/opt/scipio-erp/framework/service/data/ServiceSecurityGroupDemoData.xml
/opt/scipio-erp/framework/service/data/ServiceDemoData.xml
/opt/scipio-erp/framework/entityext/data/EntityExtSecurityGroupDemoData.xml
/opt/scipio-erp/framework/webtools/data/WebtoolsSecurityGroupDemoData.xml
/opt/scipio-erp/applications/party/data/PartySecurityGroupDemoData.xml
/opt/scipio-erp/applications/party/data/PartyDemoData.xml
/opt/scipio-erp/applications/securityext/data/UserDemoData.xml
/opt/scipio-erp/applications/content/data/ContentSecurityGroupDemoData.xml
/opt/scipio-erp/applications/content/data/UsersDemoData.xml
/opt/scipio-erp/applications/content/data/WebSitePublishPointData.xml
/opt/scipio-erp/applications/content/data/CompDocData.xml
/opt/scipio-erp/applications/workeffort/data/WorkEffortSecurityGroupDemoData.xml
/opt/scipio-erp/applications/workeffort/data/WorkEffortDemoData.xml
/opt/scipio-erp/applications/product/data/ProductSecurityGroupDemoData.xml
/opt/scipio-erp/applications/product/data/FacilitySecurityGroupDemoData.xml
/opt/scipio-erp/applications/product/data/ProductDemoData.xml
/opt/scipio-erp/applications/manufacturing/data/ManufacturingSecurityGroupDemoData.xml
/opt/scipio-erp/applications/manufacturing/data/ManufacturingExampleData.xml
/opt/scipio-erp/applications/accounting/data/AccountingSecurityGroupDemoData.xml
/opt/scipio-erp/applications/accounting/data/DemoGeneralChartOfAccounts.xml
/opt/scipio-erp/applications/accounting/data/DemoOrganizationData.xml
/opt/scipio-erp/applications/accounting/data/DemoGlSetupData.xml
/opt/scipio-erp/applications/accounting/data/DemoFinAccountData.xml
/opt/scipio-erp/applications/accounting/data/DemoTaxAuthority.xml
/opt/scipio-erp/applications/accounting/data/DemoAcctgTransactionData.xml
/opt/scipio-erp/applications/accounting/data/DemoPaymentsInvoices.xml
/opt/scipio-erp/applications/accounting/data/DemoAccountingUserData.xml
/opt/scipio-erp/applications/accounting/data/DemoAssetMaintData.xml
/opt/scipio-erp/applications/accounting/data/DemoBudgetData.xml
/opt/scipio-erp/applications/humanres/data/HumanResSecurityGroupDemoData.xml
/opt/scipio-erp/applications/humanres/data/HumanResDemoData.xml
/opt/scipio-erp/applications/order/data/OrderSecurityGroupDemoData.xml
/opt/scipio-erp/applications/order/data/OrderDemoUser.xml
/opt/scipio-erp/applications/order/data/OrderDemoData.xml
/opt/scipio-erp/applications/marketing/data/MarketingSecurityGroupDemoData.xml
/opt/scipio-erp/applications/marketing/data/sfaDemoData.xml

[demo SHOP data starts… Not part of ‘company’]
/opt/scipio-erp/applications/shop/data/DemoStoreData.xml
/opt/scipio-erp/applications/shop/data/DemoRentalStoreData.xml
/opt/scipio-erp/applications/shop/data/DemoCatalogData.xml
/opt/scipio-erp/applications/shop/data/DemoCatalogGenericProductData.xml
/opt/scipio-erp/applications/shop/data/DemoRentalCatalogData.xml
/opt/scipio-erp/applications/shop/data/DemoSurveyData.xml
/opt/scipio-erp/applications/shop/data/DemoSurveyTestData.xml
/opt/scipio-erp/applications/shop/data/DemoWebSitePublishData.xml
/opt/scipio-erp/applications/shop/data/DemoTemplateData.xml
/opt/scipio-erp/applications/shop/data/DemoContentData.xml
/opt/scipio-erp/applications/shop/data/DemoTopicData.xml
/opt/scipio-erp/applications/shop/data/DemoTreeData.xml
/opt/scipio-erp/applications/shop/data/DemoRelatedData.xml
/opt/scipio-erp/applications/shop/data/DemoCatalogConfigurationData.xml
/opt/scipio-erp/applications/shop/data/DemoCatalogI18nData.xml
/opt/scipio-erp/applications/shop/data/DemoCatalogAdditionalViewData.xml
/opt/scipio-erp/applications/shop/data/DemoTreeContentData.xml
/opt/scipio-erp/applications/shop/data/DemoContentAltUrlData.xml
/opt/scipio-erp/applications/shop/data/DemoRentalShippingData.xml
/opt/scipio-erp/applications/shop/data/DemoShippingData.xml
/opt/scipio-erp/applications/shop/data/DemoTaxAuthorityData.xml
/opt/scipio-erp/applications/shop/data/DemoOrderPartyData.xml
/opt/scipio-erp/applications/shop/data/DemoMarketingData.xml
/opt/scipio-erp/applications/shop/data/DemoPurchasingData.xml
/opt/scipio-erp/applications/shop/data/DemoStandardCostingData.xml
/opt/scipio-erp/applications/shop/data/DemoStoreGiftCertificateData.xml
/opt/scipio-erp/applications/shop/data/DemoFinAccountData.xml
/opt/scipio-erp/applications/shop/data/DemoCatalogTransientData.xml
/opt/scipio-erp/applications/shop/data/DemoCatalogImageData.xml
/opt/scipio-erp/applications/shop/data/DemoOrderTestData.xml
/opt/scipio-erp/applications/commonext/data/CommonExtSecurityGroupDemoData.xml
/opt/scipio-erp/applications/commonext/data/OfbizSetupSecurityGroupDemoData.xml
/opt/scipio-erp/specialpurpose/assetmaint/data/AssetMaintSecurityGroupDemoData.xml

The trick is to take the demo data, copy and rename (and/or eliminate) into the hot-deploy company data directory (i.e. take out ‘Demo’ in the name), and EDIT each file and cut out stuff you don’ need for YOUR company. The order is critical. This takes a long time. There should be no errors.

After you get the company data successfully loaded from hot-deploy you can then start working on the actual store… Also in hot-deploy, as a separate set of data. There is no other way. You WILL screw up the data, so you reload and start again. In the end, you should be able to wipe the DB, load seed, load company, load store(s), and you have a clean DB. No GUI, no “setup”

You’ll find out (LATER) that you may need some special user loaded, or you need some multi-company accounting setup. You go into the SEED data, edit it, reload.

If this seems to be too much, seek professional help.


#3

What an incredibly detailed post. Thank you so much for sharing, @mz4wheeler!

Perhaps we should consider adding this to the website. Would you be open for turning this into a guest article?


#4

Well, I mostly agree with all Mike said here. In my opinion though, I think that it is not always the case where you have to deal with seed data, it’s more a combination of both. For certain basic setups I am pretty sure it can be achieved entirely through the GUI (with some minor seed data touches).

Even multi-organization could be achieved partially via GUI, where setup would be your starting point, which allows to setup multiple party groups (the ones with the INTERNAL_ORGANIZATIO role). But here’s where I think Mike is totally right, you need to repeat the seed process all the time. So, in order to do that, I’d propose the following:

  1. Start with a clean installation without demo seed data (./ant load-extseed)
  2. Create new component/s as Mike stated.
  3. Create each organization through GUI and fulfill all its data as you can.
  4. When you think an organization has been filled up, go to the admin: export and dump the entire DB or select the entities you know for sure that have been populated. In the end you need to be familiar with the datamodel (check the datamodel diagrams https://www.scipioerp.com/community/developer/entities#literature for better help).
  5. Extract and separate all the useful data from those seed data files and move them to the data folder of the components you created. You can create multiple seed data files per organization if you want.
  6. In the ofbiz-component.xml, define the corresponding data files as ext, ie: <entity-resource type="data" reader-name="ext" loader="main" location="data/MyOrganization1Data.xml"/>
  7. Manually add all the stuff missing that couldn’t be achieved via GUI by editing those files, again you’ll need a proper understanding of the datamodel.

By following these steps, you’ll be able to setup multiple organizations and reseeding any time to ensure you are setting up your instance properly without messing exclusively with seed data files.


#5

I may also wanna add:

@mz4wheeler : when was the last time you checked the setup component of ours? for the current release we updated large sections of it, so i’d love to hear your feedback.


#6

Thanks @mz4wheeler, @minifreak and @madppiper for your comments, very interesting; thought it was not possible since no answer came for days. Following these process will sure take some time, but I will try it and give you feedback .
Thanks


#7

Well, there are 1000s (or in the case of zip codes) millions of items to add that would be virtually impossible to do it any other way except via generating external files. Here is a real example:

grep '<geoId>' statetax_zipcodes_plus4.xml | wc -l
3457542

Yes, 3.4 million entries… Each look like this:

<Geo>
  <geoId>USA-98001</geoId>
  <geoTypeId>POSTAL_CODE</geoTypeId>
  <geoName>98001</geoName>
  <geoCode>98001</geoCode>
  <abbreviation>98001</abbreviation>
  <createdStamp>2018-10-07 11:45:12</createdStamp>
</Geo>
<GeoAssoc>
  <geoId>WA-1702</geoId>
  <geoIdTo>USA-98001</geoIdTo>
  <geoAssocTypeId>REGIONS</geoAssocTypeId>
  <createdStamp>2018-10-07 11:45:12</createdStamp>
</GeoAssoc>

So, yes, only via external XML files. You have to have a repeatable data seeding system that will save you when you wipe out something.

Regarding the setup, no, I haven’t tried it but I assume that the demo data has most of it covered. If something is wrong, I just tweak the seed data, and reload, right?


#8

@mz4wheeler well, I think that it all comes down to how tech-savvy you really are. One of the key complaints from our clients in recent months has been that working with the system requires too much knowledge of xml based data creation. It surprised us a bit at first, but made sense when you consider that for most systems the seed data is used during the initial installation only. Afterwards people will and have to rely on database backup mechanisms and other forms of input.

The setup component is an easy way to accomplish what you are describing and is targeted towards those who want to set it all up manually on the system once. It essentially creates most of the datasets you posted anyway, so the difference are slim.

However, there is a reasonable point to be made, that for the import of large batches of data as well as for understanding the inner workings of the database tables the seed data files are better suited. Which is why i would be interested in turning what you wrote to a proper article. Apart from that, feedback to the setup component is also highly appreciated, especially since we don’t want to turn away those, who get lost in the datasets.


#9

I set up one organisation, while trying to test it, I ran into these errors: at the level of completing the production run and produce: should surely just be a configuration error. Pls how do I correct this?

Error message:
1. Error trying to begin transaction, could not process method: The current transaction is marked for rollback, not beginning a new transaction and aborting current operation; the rollbackOnly was caused by: Error in simple-method [Post a AcctgTrans [file:/H:/erp/scipio/applications/accounting/script/org/ofbiz/accounting/ledger/AcctgTransServices.xml#postAcctgTrans]]: ; [Gl account id is not set for 00001 or mapping not found for account type WIP_INVENTORY, Gl account id is not set for 00002 or mapping not found for account type INVENTORY_ACCOUNT]


#10

Go to https://localhost:8443/accounting/control/GlAccountAssignment?organizationPartyId=Company (replace Company by the organization id you created) and map the corresponding GlAccount to the WIP_INVENTORY (Work In Progress Inventory) and INVENTORY_ACCOUNT (Inventory).

Worth to mention that in the future you will be able to do these mappings during setup, not sure if that will be available for 1.14.5 though but definitely for 1.14.6.


#11

Thanks done! Let me continue my course