Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision
Previous revision
auto-import_data_feature [2019/01/30 09:22]
127.0.0.1 external edit
auto-import_data_feature [2019/09/24 18:21]
admin
Line 1: Line 1:
-======  ======+The Auto-Import Feature in Control allows users to tie Control into other programs, such as web lead programs and external contact management. The SSLIP monitors a folder and when a file is placed into it uses Control's built-in import and mapping capabilities to try and import the data.
  
 +The basic steps in the auto-import process are:
  
- 
-The Auto-Import Feature in Control allows users to tie Control into other programs, such as web lead programs and external contact management. The SSLIP monitors a folder and when a file is placed into it it uses Control's built in import and mapping capabilities to try and import the data. 
- 
- 
- 
-The basic steps in the auto-import process are: 
   - Setup the SSLIP to monitor a hot folder.   - Setup the SSLIP to monitor a hot folder.
   - Restart the SSLIP.   - Restart the SSLIP.
Line 13: Line 8:
   - Copy the map file into the MAPS subfolder.   - Copy the map file into the MAPS subfolder.
   - Drop the import files into the hot folder.   - Drop the import files into the hot folder.
- 
- 
  
 It is that simple. The rest of this document covers these steps in details. It is that simple. The rest of this document covers these steps in details.
  
- +**Note**: The Auto-Import feature was released with Control 4.1. Map files created with earlier versions will not work with the auto-import feature. To update them, open the map files in Control and resave them. This will add the additional information needed for the Auto-Import feature to work. 
- +\\ 
-<html><div style="margin-left: 20px;"> **Note**: The Auto-Import feature was released with Control 4.1. Map files created with earlier versions will not work with the auto-import feature. To update them, open the map files in Control and resave them. This will add the additional information needed for the Auto-Import feature to work. </div></html> +\\
- +
- +
 ===== Setting up the Folder Structure ===== ===== Setting up the Folder Structure =====
- 
- 
  
 When auto-import is enabled, the SSLIP monitors a "hot" folder (a.k.a. "live" folder) for any changes. When a file is placed in that folder, the SSLIP automatically tries to import the file. When auto-import is enabled, the SSLIP monitors a "hot" folder (a.k.a. "live" folder) for any changes. When a file is placed in that folder, the SSLIP automatically tries to import the file.
- 
- 
  
 When the SSLIP is setup to monitor a hot folder, it will auto create the folder structure. To do this manually, create the following folders on the same computer as the SSLIP: When the SSLIP is setup to monitor a hot folder, it will auto create the folder structure. To do this manually, create the following folders on the same computer as the SSLIP:
- 
- 
  
 <html><div style="margin-left: 20px;"> C:\Program Files\Cyrious\SSLIP\HotFolders\ImportHotFolder\Import </div></html> <html><div style="margin-left: 20px;"> C:\Program Files\Cyrious\SSLIP\HotFolders\ImportHotFolder\Import </div></html>
- 
- 
  
 You may also need to share this folder if the source of the import files is not on the same computer. If so, we recommend you name the share **ImportHotFolder$**. (The dollar sign at the end makes it a hidden share that will not show up in Windows Explorer but can still be accessed by typing the path directly.) You may also need to share this folder if the source of the import files is not on the same computer. If so, we recommend you name the share **ImportHotFolder$**. (The dollar sign at the end makes it a hidden share that will not show up in Windows Explorer but can still be accessed by typing the path directly.)
- 
- 
  
 Under the **ImportHotFolder\Import**, create four additional folders: Under the **ImportHotFolder\Import**, create four additional folders:
  
- +  * **C:\Program Files\Cyrious\SSLIP\HotFolders\ImportHotFolder\Import\Maps**<html><div style="margin-left: 20px;"> This folder is where the map files (created in the next step) are stored. You can have multiple map files for different import sources as long as you keep the naming unique. </div></html> 
- +  * **C:\Program Files\Cyrious\SSLIP\HotFolders\ImportHotFolder\Import\Logs**<html><div style="margin-left: 20px;"> Since the conversion is not monitored, any messages are written to log files and put in this folder. </div></html> 
-{{::http:www.ecyrious.comcontrolimagesauto-import_folder_path.jpg?nolink&|}} +  * **C:\Program Files\Cyrious\SSLIP\HotFolders\ImportHotFolder\Import\Completed**<html><div style="margin-left: 20px;"> Upon successful import, the SSLIP will move the import file from the Import folder into the Completed folder. </div></html> 
-  * **C:\Program Files\Cyrious\SSLIP\HotFolders\ImportHotFolder\Import\Maps** +  * **C:\Program Files\Cyrious\SSLIP\HotFolders\ImportHotFolder\Import\Failed**<html><div style="margin-left: 20px;"> Upon a failed import, the SSLIP will move the import file from the Import folder into the failed folder. An error message containing the problem will be placed in the llogs folder. </div></html>
- +
- +
- +
-<html><div style="margin-left: 20px;"> This folder is where the map files (created in the next step) are stored. You can have multiple map files for different import sources as long as you keep the naming unique. </div></html> +
-  * **C:\Program Files\Cyrious\SSLIP\HotFolders\ImportHotFolder\Import\Logs** +
- +
- +
- +
-<html><div style="margin-left: 20px;"> Since the conversion is not monitored, any messages are written to log files and put in this folder. </div></html> +
-  * **C:\Program Files\Cyrious\SSLIP\HotFolders\ImportHotFolder\Import\Completed** +
- +
- +
- +
-<html><div style="margin-left: 20px;"> Upon successful import, the SSLIP will move the import file from the Import folder into the Completed folder. </div></html> +
-  * **C:\Program Files\Cyrious\SSLIP\HotFolders\ImportHotFolder\Import\Failed** +
- +
- +
- +
-<html><div style="margin-left: 20px;"> Upon a failed import, the SSLIP will move the import file from the Import folder into the failed folder. An error message containing the problem will be placed in the llogs folder. </div></html> +
- +
- +
 Copy the map file you created above to the Maps folder. During the auto-import process, the SSLIP will automatically look for all map files in the Maps folder. Copy the map file you created above to the Maps folder. During the auto-import process, the SSLIP will automatically look for all map files in the Maps folder.
- +\\ 
- +\\
 ===== Creating a Map File ===== ===== Creating a Map File =====
- 
- 
  
 Within Control, you create the map file just like you were doing a regular import. Within Control, click on Tools | Import Data | Import from Text Files. This will open the generic import screen. Within Control, you create the map file just like you were doing a regular import. Within Control, click on Tools | Import Data | Import from Text Files. This will open the generic import screen.
  
- +Choose your import file on the right and select the type of import on the left. Now establish a mapping of the fields in both panels by selecting a field in your import document and the corresponding field in the Control tree and clicking **Add Link**. Repeat this process until you have mapped all of the desired fields in your import document to their corresponding Control fields. When you are satisfied with the mappings established, click on the **Save Mapping**  button and enter a file name.
- +
-Choose your import file on the right and select the type of import on the left. Now establish a mapping of the fields in both panels by selecting a field in your import document and the corresponding field in the Control tree and clicking **Add Link**. Repeat this process until you have mapped all of the desired fields in your import document to their corresponding Control fields. When you are satisfied with the mappings established, click on the **Save Mapping** button and enter a file name. +
- +
  
 The map file name will need to match the import file so that the auto-import can associate the mapping. (This allows you to have multiple map files working simultaneously.) The mapping file name will be The map file name will need to match the import file so that the auto-import can associate the mapping. (This allows you to have multiple map files working simultaneously.) The mapping file name will be
  
- +<html><div style="margin-left: 20px;"> [[mapfile.map]] </div></html>
- +
-<html><div style="margin-left: 20px;"> **.map** </div></html> +
- +
  
 For customers, you might name the file Customer.map. If however, you have two different sources for customer imports, you might use more descriptive names like WebCustomers.map and POSCustomers.map. Under the importing data section, you will see that you must name the import file properly so the SSLIP will find the corresponding map file. For customers, you might name the file Customer.map. If however, you have two different sources for customer imports, you might use more descriptive names like WebCustomers.map and POSCustomers.map. Under the importing data section, you will see that you must name the import file properly so the SSLIP will find the corresponding map file.
- 
- 
  
 Copy the map file to the Maps folder located at: Copy the map file to the Maps folder located at:
- 
- 
  
 C:\Program Files\Cyrious\SSLIP\HotFolders\ImportHotFolder\Import\Maps C:\Program Files\Cyrious\SSLIP\HotFolders\ImportHotFolder\Import\Maps
- +\\ 
- +\\
 ===== Setting up the SSLIP ===== ===== Setting up the SSLIP =====
- 
- 
  
 When auto-import is enabled, the SSLIP monitors a "hot" folder (a.k.a. "live" folder) for any changes. When a file is placed in that folder, the SSLIP automatically tries to import the file. When auto-import is enabled, the SSLIP monitors a "hot" folder (a.k.a. "live" folder) for any changes. When a file is placed in that folder, the SSLIP automatically tries to import the file.
  
 +To set up the SSLIP for auto-import, all you need to do is enable the hot folder using these steps:
  
- 
-To set up the SSLIP for auto-import, all you need to do is enable the hot folder using these steps: 
   - Log into the SSLIP and click on **Database Administration**.   - Log into the SSLIP and click on **Database Administration**.
   - Select the Control StoreData group and click Edit.   - Select the Control StoreData group and click Edit.
-  - Check the "Enable Hot Folder" checkbox and enter the name of the Hot Folder (**ImportHotFolder** in these examples). Note that you may not specify a full path, but only the name of the folder under the SSLIP folder.+  - Check the "Enable Hot Folder" checkbox and enter the name of the Hot Folder (**ImportHotFolder**  in these examples). Note that you may not specify a full path, but only the name of the folder under the SSLIP folder.
   - Save the setting.   - Save the setting.
   - Restart the SSLIP.   - Restart the SSLIP.
- +\\ 
- +\\
 ===== Importing Data ===== ===== Importing Data =====
- 
- 
  
 The SSLIP actively monitors the ImportHotFolder. If it finds a new file in the \Import folder, the SSLIP attempts to import the file. The processing that occurs depends upon the extension of the new file. The SSLIP actively monitors the ImportHotFolder. If it finds a new file in the \Import folder, the SSLIP attempts to import the file. The processing that occurs depends upon the extension of the new file.
- +\\ 
- +\\
 ==== TXT and CSV Files ==== ==== TXT and CSV Files ====
  
- +Import files ending with .TXT are assumed to be tab-delimited files, whereas .CSV is the traditional ending for a commas-delimited file. (Regardless of the extensionthe actual file type must be specified by the map file).
- +
-Import files ending with .TXT are assumed to be tab-delimited files, whereas .CSV is the traditional ending for a commas-delimited files. (Regardless of the extentiontha actual file type must be specified by the map file). +
- +
  
 When a new TXT or CSV import file is places in the ImportHotFolder\Import folder, the SSLIP assumes the file has a corresponding map file and should be imported. The name of the file must also contain a caret and the name of the map file, as in: When a new TXT or CSV import file is places in the ImportHotFolder\Import folder, the SSLIP assumes the file has a corresponding map file and should be imported. The name of the file must also contain a caret and the name of the map file, as in:
  
 +<html><div style="margin-left: 20px;"> [[xxxxxxx]]^[[mapfile]].txt </div></html>
  
- +<html><div style="margin-left: 20px;"> [[xxxxxxx]]^[[mapfile]].csv </div></html>
-<html><div style="margin-left: 20px;"> ^.txt </div></html> +
- +
- +
- +
-<html><div style="margin-left: 20px;"> ^.csv </div></html> +
- +
  
 where where
  
 +<html><div style="margin-left: 20px;">[[xxxxxxx]] = a name given to the file</div></html>
  
 +<html><div style="margin-left: 20px;">[[mapfile]] = the name of the map file in the MAPS folder (with an extension of .map)</div></html>
  
-<html><div style="margin-left: 20px;">  = a name given to the file </div></html> +\\
- +
- +
- +
-<html><div style="margin-left: 20px;">  = the name of the map file in the MAPS folder </div></html> +
- +
- +
- +
-(with an extension of .map) +
- +
- +
 The SSLIP then looks for the map file by the name .map in the ImportHotFolder\Imports\Maps folder. If the map file can't be located, the import will fail and the SSLIP will generate a corresponding log file. If the map file is found, the appropriate files are loaded and the import begins. The SSLIP then looks for the map file by the name .map in the ImportHotFolder\Imports\Maps folder. If the map file can't be located, the import will fail and the SSLIP will generate a corresponding log file. If the map file is found, the appropriate files are loaded and the import begins.
- 
- 
  
 The SSLIP writes the results of the import to the Results Log. If there are any errors during the import process, the SSLIP writes these errors to the Error Log and cancels the import. The SSLIP writes the results of the import to the Results Log. If there are any errors during the import process, the SSLIP writes these errors to the Error Log and cancels the import.
- +\\ 
- +\\
 ==== XML Files ==== ==== XML Files ====
- 
- 
  
 If the file ends with .XML, the SSLIP assumes it is a native Control file and tries to imports it accordingly. There are no other configuration options. If the file ends with .XML, the SSLIP assumes it is a native Control file and tries to imports it accordingly. There are no other configuration options.
- 
- 
  
 The SSLIP writes the results of the import to the Results Log. If there are any errors during the import process, the SSLIP writes these errors to the Error Log and cancels the import. The SSLIP writes the results of the import to the Results Log. If there are any errors during the import process, the SSLIP writes these errors to the Error Log and cancels the import.
- +\\ 
- +\\
 ==== TXTLINKS ==== ==== TXTLINKS ====
- 
- 
  
 If two files are to be imported in combination, such as a Customer list and a separate Contacts list, you will need to use a Text Link file. If you drop either file in the folder individually, they could be processed before the other file was written, resulting in an error. If two files are to be imported in combination, such as a Customer list and a separate Contacts list, you will need to use a Text Link file. If you drop either file in the folder individually, they could be processed before the other file was written, resulting in an error.
- 
- 
  
 If the file ends with .TXTLINKS, the SSLIP assumes the file is a text document that contains the path to a map file and one or two import files. The contents of the files must conform to the following example: If the file ends with .TXTLINKS, the SSLIP assumes the file is a text document that contains the path to a map file and one or two import files. The contents of the files must conform to the following example:
  
 +<html><div style="margin-left: 20px;">[[mapfilepath]]\[[mapfile]].link</div></html>
  
 +<html><div style="margin-left: 20px;">[[filepathA]]\[[filenameA]].txt</div></html>
  
-<html><div style="margin-left: 20px;"> \.link </div></html> +<html><div style="margin-left: 20px;">[[filepathB]]\[[filenameB]].txt</div></html>
- +
- +
- +
-<html><div style="margin-left: 20px;"> \.txt </div></html> +
- +
- +
- +
-<html><div style="margin-left: 20px;"> \.txt </div></html> +
- +
  
 where where
  
 +<html><div style="margin-left: 20px;">[[mapfilepath]] = the path of the map file.</div></html>
  
 +<html><div style="margin-left: 20px;">[[mapfilename]] = the name of the map file.</div></html>
  
-<html><div style="margin-left: 20px;">  = the path of the map file. </div></html>+<html><div style="margin-left: 20px;">[[filepathA]] = the path of the primary import file.</div></html>
  
 +<html><div style="margin-left: 20px;">[[filenameA]] = the name of the primary import file.</div></html>
  
 +<html><div style="margin-left: 20px;">[[filepathB]] = the path of the dependent import file.</div></html>
  
-<html><div style="margin-left: 20px;">  = the name of the map file. </div></html> +<html><div style="margin-left: 20px;">[[filenameB]] = the name of the dependent import file.</div></html>
- +
- +
- +
-<html><div style="margin-left: 20px;">  = the path of the primary import file. </div></html> +
- +
- +
- +
-<html><div style="margin-left: 20px;">  = the name of the primary import file. </div></html> +
- +
- +
- +
-<html><div style="margin-left: 20px;">  = the path of the dependent import file. </div></html> +
- +
- +
- +
-<html><div style="margin-left: 20px;">  = the name of the dependent import file. </div></html> +
- +
  
 The map file and first import file are required. The second import file, if specified, should be the dependent import file. An absolute path (like C:\Temp\Cust.TXT) or a UNC path (like \\Server\\Import$\Cust.TXT) is acceptable, though the TXTLINKS file itself must be located in the ImportHotFolder. The map file and first import file are required. The second import file, if specified, should be the dependent import file. An absolute path (like C:\Temp\Cust.TXT) or a UNC path (like \\Server\\Import$\Cust.TXT) is acceptable, though the TXTLINKS file itself must be located in the ImportHotFolder.
- 
- 
  
 The results and errors are logged normally. The results and errors are logged normally.
- +\\ 
- +\\
 ==== Other File Types ==== ==== Other File Types ====
- 
- 
- 
 The SSLIP ignores any other file types found in the ImportHotFolder. The SSLIP ignores any other file types found in the ImportHotFolder.
- +\\ 
- +\\
 ===== Result Logs ===== ===== Result Logs =====
- 
- 
- 
 During the import process, the results Control normally displays on screen are written into a log file located in the \LOGS folder. The file name of the log file is: During the import process, the results Control normally displays on screen are written into a log file located in the \LOGS folder. The file name of the log file is:
  
- +<html><div style="margin-left: 20px;"> yyyymmdd-hhnnresults.log </div></html>
- +
-<html><div style="margin-left: 20px;"> yyyymmdd-hhnn--results.log </div></html> +
- +
  
 where where
- 
- 
  
 <html><div style="margin-left: 20px;"> yyyy = current year </div></html> <html><div style="margin-left: 20px;"> yyyy = current year </div></html>
- 
- 
  
 <html><div style="margin-left: 20px;"> mm = current month (01-12) </div></html> <html><div style="margin-left: 20px;"> mm = current month (01-12) </div></html>
- 
- 
  
 <html><div style="margin-left: 20px;"> dd = current day (01-31) </div></html> <html><div style="margin-left: 20px;"> dd = current day (01-31) </div></html>
  
- +<html><div style="margin-left: 20px;"> [[filename]] = the file name of the import file </div></html>
- +
-<html><div style="margin-left: 20px;">  = the file name of the import file </div></html> +
- +
  
 This file should be deleted by the application which generated the import file, or by the user at least monthly. This file should be deleted by the application which generated the import file, or by the user at least monthly.
- +\\ 
- +\\
 ===== Error Logs ===== ===== Error Logs =====
- 
- 
- 
 Because the SSLIP is an unmanned program, user intervention is not possible Any errors in the import process are written to an error log located in the \LOGS folder. The file name of the error file is: Because the SSLIP is an unmanned program, user intervention is not possible Any errors in the import process are written to an error log located in the \LOGS folder. The file name of the error file is:
  
- +<html><div style="margin-left: 20px;"> yyyymmdd-hhnnerror.log </div></html>
- +
-<html><div style="margin-left: 20px;"> yyyymmdd-hhnn--error.log </div></html> +
- +
  
 where where
- 
- 
  
 <html><div style="margin-left: 20px;"> yyyy = current year </div></html> <html><div style="margin-left: 20px;"> yyyy = current year </div></html>
- 
- 
  
 <html><div style="margin-left: 20px;"> mm = current month (01-12) </div></html> <html><div style="margin-left: 20px;"> mm = current month (01-12) </div></html>
- 
- 
  
 <html><div style="margin-left: 20px;"> dd = current day (01-31) </div></html> <html><div style="margin-left: 20px;"> dd = current day (01-31) </div></html>
  
- +<html><div style="margin-left: 20px;"> [[filename]] = the file name of the import file </div></html>
- +
-<html><div style="margin-left: 20px;">  = the file name of the import file </div></html> +
- +
  
 There are no warnings within Control, and it is up to the source of the import file to handle any errors. After processing the error, this file should be deleted by the application which generated the import file, or by the user at least monthly. There are no warnings within Control, and it is up to the source of the import file to handle any errors. After processing the error, this file should be deleted by the application which generated the import file, or by the user at least monthly.
- +\\ 
- +\\
 ===== Removing the Import Files ===== ===== Removing the Import Files =====
- 
- 
- 
 After the import, the imported file is moved out of the ImportHotFolder. This keeps the import folder free of processed items. For TXTLINKS files, the TXTLINKS file is moved but the referenced data files are not. After the import, the imported file is moved out of the ImportHotFolder. This keeps the import folder free of processed items. For TXTLINKS files, the TXTLINKS file is moved but the referenced data files are not.
  
- +If the import was successful, the file is moved into the **ImportHotFolder\Completed**  folder. If the import generated an error, the import is canceled and the file is moved to the **ImportHotFolder\Failed**  folder.
- +
-If the import was successful, the file is moved into the **ImportHotFolder\Completed** folder. If the import generated an error, the import is cancelled and the file is moved to the **ImportHotFolder\Failed** folder. +
- +
  
 Periodically, you will want to remove the old import files from these folders. Periodically, you will want to remove the old import files from these folders.
- 
-