Python on Aamnah - Frontend Web Developer

Script for creating Shopify Product import CSV file with Python

Sun, 07 Feb 2021 15:34:42 +0500

But why?

You can have default values pre-populated
You can create a CSv file ready for import right after you have scraped some data
Why not?

This script gives you an overview of writing a CSV file with Python. I’m using a dictionary (array data) to populate the sheet

  1# -------------------------------------
  2# CSV
  3# -------------------------------------
  4
  5# Write these to a CSV file
  6filename = 'vivo_phones_shopify.csv'
  7
  8# vivo_phones_shopify.csv file will be created in the current working
  9with open( filename, 'w', newline='') as file:
 10  # filednames = a list object which should contain the column headers specifying the order in which data should be written in the CSV file
 11  fieldnames = ['Handle','Title','Body (HTML)','Vendor','Type','Tags','Published','Option1 Name','Option1 Value','Option2 Name','Option2 Value','Option3 Name','Option3 Value','Variant SKU','Variant Grams','Variant Inventory Tracker','Variant Inventory Qty','Variant Inventory Policy','Variant Fulfillment Service','Variant Price','Variant Compare At Price','Variant Requires Shipping','Variant Taxable','Variant Barcode','Image Src','Image Position','Image Alt Text','Gift Card','SEO Title','SEO Description','Google Shopping / Google Product Category','Google Shopping / Gender','Google Shopping / Age Group','Google Shopping / MPN','Google Shopping / AdWords Grouping','Google Shopping / AdWords Labels','Google Shopping / Condition','Google Shopping / Custom Product','Google Shopping / Custom Label 0','Google Shopping / Custom Label 1','Google Shopping / Custom Label 2','Google Shopping / Custom Label 3','Google Shopping / Custom Label 4','Variant Image','Variant Weight Unit','Variant Tax Code','Cost per item','Status']
 12  writer = csv.DictWriter(file, fieldnames = fieldnames)
 13  # writer = csv.writer(file, fieldnames = fieldnames)
 14
 15  writer.writeheader()
 16
 17  for title, price, image in zip(titles, prices, images):
 18    # Basics
 19    title = title.text.strip()
 20    price = price.text.strip()
 21    image = image.get('src').strip()
 22
 23    # Shopify defaults
 24    handle = title.lower().replace(' ', '_' ) # Required. Can't be blank or missing.
 25    # title = '' # Required. Can't be blank or missing.
 26    body_html = '' # Required, but can be blank.
 27    vendor = 'Vivo' # Required, but can be blank.
 28    product_type = 'Smartphone' # Required, but can be blank.
 29    tags = '' # Required, but can be blank.
 30    published = 'TRUE' # Required. Leaving the field blank publishes the product.
 31    option1_name = 'Title' # Required. Can't be blank or missing. For products with only one option, this should be set to 'Title'.
 32    option1_value = 'Default Title' # Required. Can't be blank or missing. For products with only one option, this should be set to 'Default Title'.
 33    option2_name = '' # Required, but can be blank.
 34    option2_value = '' # Required, but can be blank.
 35    option3_name = '' # Required, but can be blank.
 36    option3_value = '' # Required, but can be blank.
 37    variant_sku = '' # Required, but can be blank.
 38    variant_grams = '350' # Required. Can't be blank or missing. You must enter a value, even if that value is '0'.
 39    variant_inventory_tracker = '' # Required, but can be blank.
 40    variant_inventory_qty = '' # Required, but can be blank.
 41    variant_inventory_policy = 'continue' # Required. Can't be blank or missing. (values: continue, deny). If the existing inventory tracking options are removed, then inventory is no longer tracked.
 42    variant_fulfillment_service = 'manual' # Required. Can't be blank or missing. (values: manual, shipwire, amazon_marketplace_web etc.)
 43    variant_price = price # Required. Can't be blank or missing.
 44    variant_compare_at_price = '' # Required, but can be blank.
 45    variant_requires_shipping = 'TRUE' # Required, but can be blank.
 46    variant_taxable = 'FALSE' # Required, but can be blank.
 47    variant_barcode = '' # Required, but can be blank.
 48    image_src = image # Required, but can be blank.
 49    image_position = '' # Required, but can be blank.
 50    image_alt_text = '' # Required, but can be blank.
 51    gift_card = 'FALSE' # Required, but can be blank.
 52    seo_title = '' # Optional (70 chars)
 53    seo_description = '' # Optional (320 chars)
 54    google_shopping_google_product_category = '' # Optional
 55    google_shopping_gender = '' # Optional
 56    google_shopping_age_group = '' # Optional
 57    google_shopping_mpn = '' # Optional
 58    google_shopping_adwords_grouping = '' # Optional
 59    google_shopping_adwords_labels = '' # Optional
 60    google_shopping_condition = '' # Optional
 61    google_shopping_custom_product = '' # Optional
 62    google_shopping_custom_label_0 = '' # Optional
 63    google_shopping_custom_label_1 = '' # Optional
 64    google_shopping_custom_label_2 = '' # Optional
 65    google_shopping_custom_label_3 = '' # Optional
 66    google_shopping_custom_label_4 = '' # Optional
 67    variant_image = '' # Optional
 68    variant_weight_unit = 'g' # Optional (default is 'kg')
 69    variant_tax_code = '' # Optional. Available to: Shopify Plus plan
 70    cost_per_item = '' # Optional
 71    status = 'draft' # Required (active, draft, archived)
 72
 73    # Shopify CSV
 74    writer.writerow({
 75      'Handle': handle,
 76      'Title': title,
 77      'Body (HTML)': body_html,
 78      'Vendor': vendor,
 79      'Type': product_type,
 80      'Tags': tags,
 81      'Published': published,
 82      'Option1 Name': option1_name,
 83      'Option1 Value': option1_value,
 84      'Option2 Name': option2_name,
 85      'Option2 Value': option2_value,
 86      'Option3 Name': option3_name,
 87      'Option3 Value': option3_value,
 88      'Variant SKU': variant_sku,
 89      'Variant Grams': variant_grams,
 90      'Variant Inventory Tracker': variant_inventory_tracker,
 91      'Variant Inventory Qty': variant_inventory_qty,
 92      'Variant Inventory Policy': variant_inventory_policy,
 93      'Variant Fulfillment Service': variant_fulfillment_service,
 94      'Variant Price': variant_price,
 95      'Variant Compare At Price': variant_compare_at_price,
 96      'Variant Requires Shipping': variant_requires_shipping,
 97      'Variant Taxable': variant_taxable,
 98      'Variant Barcode': variant_barcode,
 99      'Image Src': image_src,
100      'Image Position': image_position,
101      'Image Alt Text': image_alt_text,
102      'Gift Card': gift_card,
103      'SEO Title': seo_title,
104      'SEO Description': seo_description,
105      'Google Shopping / Google Product Category': google_shopping_google_product_category,
106      'Google Shopping / Gender': google_shopping_gender,
107      'Google Shopping / Age Group': google_shopping_age_group,
108      'Google Shopping / MPN': google_shopping_mpn,
109      'Google Shopping / AdWords Grouping': google_shopping_adwords_grouping,
110      'Google Shopping / AdWords Labels': google_shopping_adwords_labels,
111      'Google Shopping / Condition': google_shopping_condition,
112      'Google Shopping / Custom Product': google_shopping_custom_product,
113      'Google Shopping / Custom Label 0': google_shopping_custom_label_0,
114      'Google Shopping / Custom Label 1': google_shopping_custom_label_1,
115      'Google Shopping / Custom Label 2': google_shopping_custom_label_2,
116      'Google Shopping / Custom Label 3': google_shopping_custom_label_3,
117      'Google Shopping / Custom Label 4': google_shopping_custom_label_4,
118      'Variant Image': variant_image,
119      'Variant Weight Unit': variant_weight_unit,
120      'Variant Tax Code': variant_tax_code,
121      'Cost per item': cost_per_item,
122      'Status' : status,
123    })

Scraping with Python and BeautifulSoup

Sun, 07 Feb 2021 06:06:42 +0500

Find all H2 headings with a class of `title`

 1import requests
 2from bs4 import BeautifulSoup
 3
 4url = 'https://www.samplestore.pk/search/?keywords=amazfit&dataBi=k1'
 5
 6page = requests.get(url)
 7soup = BeautifulSoup(page.content, 'html.parser')
 8
 9# find all <h3> with a class of 'title'
10# titles = soup.find_all('h2', {"class": "title"})
11
12# As of Beautiful Soup 4.1.2, you can search by CSS class using the keyword argument class_:
13titles = soup.find_all("h2", class_="title")
14
15for title in titles:
16  print(title)

 1<h2 class="title">
 2  <a href="amazfit-stratos-multisport-gps-smartwatch.html" title="">Amazfit Stratos Multisport GPS...</a>
 3</h2>
 4<h2 class="title"><a href="amazfit-gtr-smartwatch.html" title="">Amazfit GTR Smartwatch</a></h2>
 5<h2 class="title"><a href="amazfit-gts-sports-smart-watch.html" title="">Amazfit GTS Sports Smart Watch</a></h2>
 6<h2 class="title"><a href="amazfit-bip-lite-smartwatch.html" title="">Amazfit Bip Lite Smartwatch</a></h2>
 7<h2 class="title"><a href="amazfit-gts-gtr-charging-cable.html" title="">Amazfit GTS/GTR Charging Cable</a></h2>
 8<h2 class="title"><a href="amazfit-t-rex.html" title="">Amazfit T-Rex Military Smart W...</a></h2>
 9<h2 class="title"><a href="amazfit-airrun-treadmill.html" title="">Amazfit AirRun Treadmill</a></h2>
10<h2 class="title"><a href="amazfit-neo.html" title="">Amazfit Neo Retro Design</a></h2>
11<h2 class="title"><a href="amazfit-gtr-2.html" title="">Amazfit GTR 2</a></h2>
12<h2 class="title"><a href="amazfit-gts-2.html" title="">Amazfit GTS 2 Smart Watch</a></h2>
13<h2 class="title"><a href="amazfit-bip-u.html" title="">Amazfit Bip U</a></h2>

`select` vs `find_all`

i prefer select over find_all because select and it’s CSS like syntax is familiar to me as a web developer.
find is a bit clunky when it comes to finding nested elements.
select returns an array, even if there is only one result

1titles = soup.find_all("h2", class_="title")

 1<h2 class="title">
 2  <a href="amazfit-stratos-multisport-gps-smartwatch.html" title="">Amazfit Stratos Multisport GPS...</a>
 3</h2>
 4<h2 class="title"><a href="amazfit-gtr-smartwatch.html" title="">Amazfit GTR Smartwatch</a></h2>
 5<h2 class="title"><a href="amazfit-gts-sports-smart-watch.html" title="">Amazfit GTS Sports Smart Watch</a></h2>
 6<h2 class="title"><a href="amazfit-bip-lite-smartwatch.html" title="">Amazfit Bip Lite Smartwatch</a></h2>
 7<h2 class="title"><a href="amazfit-gts-gtr-charging-cable.html" title="">Amazfit GTS/GTR Charging Cable</a></h2>
 8<h2 class="title"><a href="amazfit-t-rex.html" title="">Amazfit T-Rex Military Smart W...</a></h2>
 9<h2 class="title"><a href="amazfit-airrun-treadmill.html" title="">Amazfit AirRun Treadmill</a></h2>
10<h2 class="title"><a href="amazfit-neo.html" title="">Amazfit Neo Retro Design</a></h2>
11<h2 class="title"><a href="amazfit-gtr-2.html" title="">Amazfit GTR 2</a></h2>
12<h2 class="title"><a href="amazfit-gts-2.html" title="">Amazfit GTS 2 Smart Watch</a></h2>
13<h2 class="title"><a href="amazfit-bip-u.html" title="">Amazfit Bip U</a></h2>

1titles = soup.select('h2.title > a')

<a href="amazfit-stratos-multisport-gps-smartwatch.html" title="">Amazfit Stratos Multisport GPS...</a>
<a href="amazfit-gtr-smartwatch.html" title="">Amazfit GTR Smartwatch</a>
<a href="amazfit-gts-sports-smart-watch.html" title="">Amazfit GTS Sports Smart Watch</a>
<a href="amazfit-bip-lite-smartwatch.html" title="">Amazfit Bip Lite Smartwatch</a>
<a href="amazfit-gts-gtr-charging-cable.html" title="">Amazfit GTS/GTR Charging Cable</a>
<a href="amazfit-t-rex.html" title="">Amazfit T-Rex Military Smart W...</a>
<a href="amazfit-airrun-treadmill.html" title="">Amazfit AirRun Treadmill</a>
<a href="amazfit-neo.html" title="">Amazfit Neo Retro Design</a>
<a href="amazfit-gtr-2.html" title="">Amazfit GTR 2</a>
<a href="amazfit-gts-2.html" title="">Amazfit GTS 2 Smart Watch</a>
<a href="amazfit-bip-u.html" title="">Amazfit Bip U</a>

Scraping data

extract data from a page/site
- title
- price
- image
- description
- stock status (sold out, in stock)
- discounts (price, discounted price, is there a promotion running on this product)
create a CSV file with that data
Crawl through all Product pages on the site and populate a spreadsheet
Schedule the crawling to run on a frequency (e.g. once a day)
How to find nested tags?

Asset management in Flask

Mon, 01 Sep 2014 00:00:01 +0500

tags: css and javascript optimization

How to merge, minify and compile the code of your web assets in Flask

webassets is an asset management library for Python. Flask-Assets is an integration of webassets in your Flask application.

While asset management in itself is pretty neat, the best part is that you can set it to automatically merge, minify, and compile your asset files like CSS and JavaScript.

For example, if you work with Sass, you can set webassets to watch the sass file, merge multiple generated stylesheets into one and then minify it all and include that minified file in the project. Same goes for JavaScript files.

virtualenvwrapper

Fri, 29 Aug 2014 00:00:00 +0500

Docs

Features

Organizes all of your virtual environments in one place.
Wrappers for managing your virtual environments (create, delete, copy).
Use a single command to switch between environments.
Tab completion for commands that take a virtual environment as argument.
User-configurable hooks for all operations (see Per-User Customization). 6.Plugin system for more creating sharable extensions (see Extending Virtualenvwrapper).

install:

1pip install virtualenvwrapper

setting up

We need to specify the directory that’ll be the home of all of our virtual environments. To do that we need to set an environment variable WORKON_HOME and set it’s value to our virtualenv home directory. We also need to source the actual virtualenvwrapper shell script that’ll allow us to run virtualenvwrapper commands.

web scraping

Mon, 18 Aug 2014 00:00:00 +0500