Welcome to CSV Importer’s documentation!¶
Contents: CSV importer is a tool which allow you to transform easily a csv file into a python object or a django model instance. It is based on a django-style declarative model.
Simple, like Pypi package:
or with Pip
pip install csvImporter
Here is a basic sample:
>>> class MyCsvModel(CsvModel): >>> name = CharField() >>> age = IntegerField() >>> length = FloatField() >>> >>> class Meta: >>> delimiter = ";"
You declare a MyCsvModel which will match to a csv file like this: “Anthony;27;1.75”
To import the file or any iterable object, just do:
>>> my_csv_list = MyCsvModel.import_data(data = open("my_csv_file_name.csv")) >>> first_line = my_csv_list >>> first_line.age 27
Without an explicit declaration, data and columns are matched in the same order:
- Anthony –> Column 0 –> Field 0 –> name
- 27 –> Column 1 –> Field 1 –> age
- 1.75 –> Column 2 –> Field 2 –> length
If you now want to interact with a django model, you just have to add a dbModel option to the class meta.
>>> from model import CsvModel >>> >>> class MyCSvModel(CsvModel): >>> name = CharField() >>> age = IntegerField() >>> length = FloatField() >>> >>> class Meta: >>> delimiter = ";" >>> dbModel = Person
That will automatically match to the following django model.
>>> class Person(models.Model): >>> name = CharField(max_length = 100) >>> age = IntegerField() >>> length = FloatField()
If field names of your Csv model does not match the field names of your django model, you can manage this with the match keyword:
>>> class MyCSvModel(CsvModel): >>> fullname = CharField(match = "name") ...
If you don’t want to have to re-declare a CSV model whereas the Django model already exist, use a CsvDbModel.
>>> from my_projects.models import Person >>> from csvImporter.model import CsvDbModel >>> >>> class MyCsvModel(CsvDbModel): >>> >>> class Meta: >>> dbModel = Person >>> delimiter = ";"
The django model should be imported in the model
Fields available are:
- IntegerField : return an int
- FloatField : return a float
- CharField : return a string
- ForeignKey : return a django model object
- IgnoredField : skip the value
- ComposedKeyForeign : return a django model object retrieve with multiple values as keys.
- BooleanField : return a boolean
You can give, as argument, the following options:
- define the position in the file for this field.
- define the django model name matching this field. If a list is defined, all the field matching will received the value.
- Apply the function before returning the result.
- Apply the function on the raw value (still a string).
- A class which should implement a validate function: def validate(self, value): and return a Boolean. This allow to apply some business validation on the object before uploading.
- Allow a field to read as many values as the number of remaining data on the line.
- A list of fields which composed the key. Only for ComposedKeyForeign.
- a function which determine when a boolean is True. Only for BooleanField.
Here is an example of a way to use the transform attribute.
>>> class MyCsvModel(CsvModel): >>> >>> user = ForeignKey(transform = lambda user: user.username)
ForeignKey has an additional argument:
- allow you to define on which value the object will be retrieved.
You can also skip a row during prepare, transform or in a validator by raising a SkipRow exception.
- define the delimiter of the csv file. If you do not set one, the sniffer will try yo find one itself.
- Skip the first line if True.
- If defined, the importer will create an instance of this model.
- If set to True, an error in a imported line will not stop the loading.
- CsvDbModel only. To do take into account the django field of the django model defined in this list.
Set it to LinearLayout ( by default ) or Tabular Layout. Modify the way your data are organised in.22 the file. Tabular read:B1 B2 B3
A1 C1 C2 C3 A2 C4 C5 C6 –> (A1,B1,C1), (A1,B2,C2), (A1,B3,C3), (A2,B1,C4) ... A3 C7 C8 C9
- Set as a dictionnary with the ‘keys’ value defining the list of ‘natural keys’. If the value is found, update instead of creating a new object. If the value is not found, create a new object.
When importing data, you can add an optional argument extra_fields which is a string or a list. This allow to add a value to any line of the csv file before the loading.
If you want to create more than object by line, you should use a group CSV model. This object will create the object in the same order than the csv_models attribute provided.
- list of csv model, processed in the same order than the list